[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:01 dist-test-slave-2d19 krb5kdc[6709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:01 dist-test-slave-2d19 krb5kdc[6709](info): set up 2 sockets
Aug 07 18:38:01 dist-test-slave-2d19 krb5kdc[6709](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 18:38:02.720745 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.363s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:02 dist-test-slave-2d19 krb5kdc[6709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:02.860121 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:39299
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:36379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:39299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:02.976673 6725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:02.977049 6725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:02.977142 6725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:02.981536 6725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:02.981628 6725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:02.981666 6725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:02.981742 6725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:02.981781 6725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:02.987063 6725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:36379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:39299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:39299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:02.988219 6725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:02.989320 6725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:02.998354 6738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:02.998371 6735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:02.998371 6734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:02.998438 6725 server_base.cc:1034] running on GCE node
I20260807 18:38:02.999290 6725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:03.000490 6725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:03.001708 6725 hybrid_clock.cc:648] HybridClock initialized: now 1786127883001722 us; error 61 us; skew 500 ppm
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:03.005082 6725 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:03.006079 6725 webserver.cc:460] Webserver started at http://127.6.138.126:46201/ using document root <none> and password file <none>
I20260807 18:38:03.006690 6725 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:03.006788 6725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:03.007047 6725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:03.009805 6725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "50173145885841eb9026ca95612ac6b0"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "f0145890f069fa483ec0708883558efc"
server_key_iv: "4d817ddb9bff0f02eb7c03a2977c2793"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.010412 6725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "50173145885841eb9026ca95612ac6b0"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "f0145890f069fa483ec0708883558efc"
server_key_iv: "4d817ddb9bff0f02eb7c03a2977c2793"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.014385 6725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:03.016992 6751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.018347 6725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:38:03.018508 6725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "50173145885841eb9026ca95612ac6b0"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "f0145890f069fa483ec0708883558efc"
server_key_iv: "4d817ddb9bff0f02eb7c03a2977c2793"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.018632 6725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:03.037529 6725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:03.040788 6725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:03.041013 6725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:03.048799 6725 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:39299
I20260807 18:38:03.048831 6846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:39299 every 8 connection(s)
I20260807 18:38:03.049808 6725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:03.052732 6849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:03.058557 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 6725
I20260807 18:38:03.058619 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0: Bootstrap starting.
I20260807 18:38:03.058694 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:03.060015 6697 external_mini_cluster.cc:1428] Setting key da3e72bada43d06214ea5aa2a97fa4d6
I20260807 18:38:03.061335 6849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:03.062124 6849 log.cc:824] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:03.064366 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0: No bootstrap required, opened a new log
I20260807 18:38:03.068455 6849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50173145885841eb9026ca95612ac6b0" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39299 } }
I20260807 18:38:03.068763 6849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:03.068820 6849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50173145885841eb9026ca95612ac6b0, State: Initialized, Role: FOLLOWER
I20260807 18:38:03.069314 6849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50173145885841eb9026ca95612ac6b0" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39299 } }
I20260807 18:38:03.069459 6849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:03.069561 6849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:03.069680 6849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:03.070649 6849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50173145885841eb9026ca95612ac6b0" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39299 } }
I20260807 18:38:03.070992 6849 leader_election.cc:302] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50173145885841eb9026ca95612ac6b0; no voters:
I20260807 18:38:03.071329 6849 leader_election.cc:288] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:03.071398 6855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:03.071632 6855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [term 1 LEADER]: Becoming Leader. State: Replica: 50173145885841eb9026ca95612ac6b0, State: Running, Role: LEADER
I20260807 18:38:03.071949 6855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50173145885841eb9026ca95612ac6b0" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39299 } }
I20260807 18:38:03.072719 6849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:03.073714 6856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50173145885841eb9026ca95612ac6b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50173145885841eb9026ca95612ac6b0" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39299 } } }
I20260807 18:38:03.073848 6856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:03.075249 6868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:03.075191 6857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50173145885841eb9026ca95612ac6b0. Latest consensus state: current_term: 1 leader_uuid: "50173145885841eb9026ca95612ac6b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50173145885841eb9026ca95612ac6b0" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39299 } } }
I20260807 18:38:03.075414 6857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:03.078541 6868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:03.084656 6868 catalog_manager.cc:1347] Generated new cluster ID: 018dac1233ca4939998c5c34f5d51436
I20260807 18:38:03.084735 6868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:03.087112 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:03.063261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36930 (local address 127.6.138.126:39299)
0807 18:38:03.063758 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:03.063778 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:03.065048 (+ 1270us) server_negotiation.cc:407] Connection header received
0807 18:38:03.068032 (+ 2984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:03.068062 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:03.068483 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:03.068866 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:03.071623 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.072955 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.074696 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.075478 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.075649 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:03.080027 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:03.080036 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:03.080053 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:03.080081 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:03.082265 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.082940 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.082947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.082952 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.083143 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.083592 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.083596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.083597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.084277 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:03.086040 (+ 1763us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:03.086158 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 18:38:03.086397 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":137,"threads_started":1}
I20260807 18:38:03.106132 6868 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:03.107470 6868 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:03.116804 6868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50173145885841eb9026ca95612ac6b0: Generated new TSK 0
I20260807 18:38:03.117587 6868 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:03.155675 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39299
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:36379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:03.307878 6890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:03.308156 6890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:03.308202 6890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:03.312206 6890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:03.312285 6890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:03.312346 6890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:03.317499 6890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:36379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39299
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:03.318575 6890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:03.319547 6890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:03.330004 6907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:03.330004 6904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:03.330142 6890 server_base.cc:1034] running on GCE node
W20260807 18:38:03.330005 6905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:03.330739 6890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:03.331359 6890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:03.332520 6890 hybrid_clock.cc:648] HybridClock initialized: now 1786127883332502 us; error 29 us; skew 500 ppm
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:03.335563 6890 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:03.336519 6890 webserver.cc:460] Webserver started at http://127.6.138.65:44887/ using document root <none> and password file <none>
I20260807 18:38:03.337071 6890 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:03.337133 6890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:03.337306 6890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:03.339341 6890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "52571c362a0a4a6d96403974dd052226"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "9706e702d8c3b00d8b0ba6842e39afc0"
server_key_iv: "160919af674f127eb204727db696eab9"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.339885 6890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "52571c362a0a4a6d96403974dd052226"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "9706e702d8c3b00d8b0ba6842e39afc0"
server_key_iv: "160919af674f127eb204727db696eab9"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.343894 6890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:38:03.346733 6915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.348088 6890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:38:03.348235 6890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "52571c362a0a4a6d96403974dd052226"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "9706e702d8c3b00d8b0ba6842e39afc0"
server_key_iv: "160919af674f127eb204727db696eab9"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.348408 6890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:03.357645 6890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:03.361269 6890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:03.361469 6890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:03.362073 6890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:03.362962 6890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:03.363027 6890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.363108 6890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:03.363148 6890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.373178 6890 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:35231
I20260807 18:38:03.373202 7086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:35231 every 8 connection(s)
I20260807 18:38:03.374305 6890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:03.384191 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 6890
I20260807 18:38:03.384347 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:03.384624 6697 external_mini_cluster.cc:1428] Setting key bd2ccd28f2e99a27a1218cae041385ea
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:03.390341 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:03.376553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:52057 (local address 127.6.138.126:39299)
0807 18:38:03.376701 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:03.376705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:03.377641 (+ 936us) server_negotiation.cc:407] Connection header received
0807 18:38:03.379097 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:03.379110 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:03.379176 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:03.379254 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:03.380514 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.381070 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.382232 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.382492 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.382613 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:03.385634 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:03.385638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:03.385641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:03.385668 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:03.387542 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.388314 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.388318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.388320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.388416 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.388896 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.388903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.388907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.389189 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:03.389306 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:03.389868 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 18:38:03.390093 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:38:03.390974 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:03.376978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39299 (local address 127.6.138.65:52057)
0807 18:38:03.377467 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:03.377512 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:03.378870 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:03.379430 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:03.379437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:03.379818 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:03.380367 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:03.380384 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.381217 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.381220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:03.382096 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:03.382106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.382682 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.382688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:03.382848 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:03.383503 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:03.383520 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:03.385386 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:03.387688 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:03.387698 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:03.387718 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:03.388117 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:03.388559 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:03.388565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:03.388570 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:03.388788 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:03.389308 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:03.389314 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:03.389580 (+ 266us) client_negotiation.cc:769] Sending connection context
0807 18:38:03.389767 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 18:38:03.390043 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":124,"threads_started":1}
I20260807 18:38:03.392452 7089 heartbeater.cc:347] Connected to a master server at 127.6.138.126:39299
I20260807 18:38:03.392747 7089 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:03.393419 7089 heartbeater.cc:511] Master 127.6.138.126:39299 requested a full tablet report, sending...
I20260807 18:38:03.395237 6778 ts_manager.cc:194] Registered new tserver with Master: 52571c362a0a4a6d96403974dd052226 (127.6.138.65:35231)
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:38:03.396503 6778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:52057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:03.419167 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39299
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:36379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:03.542212 7098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:03.542537 7098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:03.542623 7098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:03.546662 7098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:03.546768 7098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:03.546883 7098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:03.552487 7098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:36379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39299
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:03.553617 7098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:03.554667 7098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:03.565178 7111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:03.565194 7109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:03.565194 7107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:03.565358 7098 server_base.cc:1034] running on GCE node
I20260807 18:38:03.565902 7098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:03.566533 7098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:03.567718 7098 hybrid_clock.cc:648] HybridClock initialized: now 1786127883567702 us; error 27 us; skew 500 ppm
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:03.571118 7098 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:03.572202 7098 webserver.cc:460] Webserver started at http://127.6.138.66:40797/ using document root <none> and password file <none>
I20260807 18:38:03.572858 7098 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:03.573061 7098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:03.573352 7098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:03.575913 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "3287106813c24bdcbaa34c369c664d6f"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "a90ce18557662feed5f8d76369f8557a"
server_key_iv: "dedfbe869a82f51de99fb1c17758b827"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.576617 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "3287106813c24bdcbaa34c369c664d6f"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "a90ce18557662feed5f8d76369f8557a"
server_key_iv: "dedfbe869a82f51de99fb1c17758b827"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.580837 7098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:38:03.583535 7125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.584939 7098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:03.585095 7098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "3287106813c24bdcbaa34c369c664d6f"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "a90ce18557662feed5f8d76369f8557a"
server_key_iv: "dedfbe869a82f51de99fb1c17758b827"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.585222 7098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:03.601099 7098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:03.604761 7098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:03.605000 7098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:03.605669 7098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:03.606587 7098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:03.606674 7098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.606756 7098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:03.606819 7098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.617046 7098 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:45013
I20260807 18:38:03.617079 7317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:45013 every 8 connection(s)
I20260807 18:38:03.618021 7098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:03.627380 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7098
I20260807 18:38:03.627481 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:03.627748 6697 external_mini_cluster.cc:1428] Setting key 8326cbaf7d4c05c4ffd2fd4943d27f50
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:03.632486 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:03.620073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:53765 (local address 127.6.138.126:39299)
0807 18:38:03.620304 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:03.620309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:03.621269 (+ 960us) server_negotiation.cc:407] Connection header received
0807 18:38:03.622722 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:03.622724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:03.622775 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:03.622861 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:03.624125 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.624707 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.625846 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.626078 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.626166 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:03.628747 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:03.628750 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:03.628752 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:03.628775 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:03.630300 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.630853 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.630856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.630859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.630944 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.631293 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.631298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.631299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.631487 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:03.631594 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:03.632136 (+ 542us) server_negotiation.cc:299] Negotiation successful
0807 18:38:03.632311 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:38:03.633550 7321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:03.620609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39299 (local address 127.6.138.66:53765)
0807 18:38:03.621094 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:03.621153 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:03.622505 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:03.623031 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:03.623038 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:03.623476 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:03.623969 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:03.623982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.624835 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.624838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:03.625723 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:03.625733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.626218 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.626221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:03.626368 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:03.626935 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:03.626951 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:03.628558 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:03.630441 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:03.630447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:03.630462 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:03.630739 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:03.631040 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:03.631043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:03.631045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:03.631192 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:03.631617 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:03.631624 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:03.631879 (+ 255us) client_negotiation.cc:769] Sending connection context
0807 18:38:03.632078 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 18:38:03.632441 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":131,"threads_started":1}
I20260807 18:38:03.634790 7319 heartbeater.cc:347] Connected to a master server at 127.6.138.126:39299
I20260807 18:38:03.635080 7319 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:03.635579 7319 heartbeater.cc:511] Master 127.6.138.126:39299 requested a full tablet report, sending...
I20260807 18:38:03.636715 6778 ts_manager.cc:194] Registered new tserver with Master: 3287106813c24bdcbaa34c369c664d6f (127.6.138.66:45013)
I20260807 18:38:03.637316 6778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:53765
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:03.659979 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39299
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:36379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:03.777230 7324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:03.777526 7324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:03.777617 7324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:03.781709 7324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:03.781796 7324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:03.781893 7324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:03.787209 7324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:36379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39299
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:03.788326 7324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:03.789428 7324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:03.799965 7341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:03.799978 7336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:03.800212 7324 server_base.cc:1034] running on GCE node
W20260807 18:38:03.799968 7337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:03.800881 7324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:03.801680 7324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:03.802950 7324 hybrid_clock.cc:648] HybridClock initialized: now 1786127883802932 us; error 28 us; skew 500 ppm
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:03.805982 7324 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:03.806898 7324 webserver.cc:460] Webserver started at http://127.6.138.67:42411/ using document root <none> and password file <none>
I20260807 18:38:03.807552 7324 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:03.807648 7324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:03.807891 7324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:03.810056 7324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "66cd8d6a5c7748f4aef55bd87ac0b20c"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "a94e10647d1ed7f911428e55d7512671"
server_key_iv: "ba7e416357978fc753f424d339eb6b1e"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.810647 7324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "66cd8d6a5c7748f4aef55bd87ac0b20c"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "a94e10647d1ed7f911428e55d7512671"
server_key_iv: "ba7e416357978fc753f424d339eb6b1e"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.814522 7324 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 18:38:03.817008 7359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.818222 7324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:38:03.818360 7324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "66cd8d6a5c7748f4aef55bd87ac0b20c"
format_stamp: "Formatted at 2026-08-07 18:38:03 on dist-test-slave-2d19"
server_key: "a94e10647d1ed7f911428e55d7512671"
server_key_iv: "ba7e416357978fc753f424d339eb6b1e"
server_key_version: "encryptionkey@0"
I20260807 18:38:03.818478 7324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:03.833792 7324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:03.836692 7324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:03.836892 7324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:03.837500 7324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:03.838346 7324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:03.838420 7324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.838486 7324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:03.838539 7324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:03.847893 7324 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:44253
I20260807 18:38:03.847935 7552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:44253 every 8 connection(s)
I20260807 18:38:03.848925 7324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:03.857602 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7324
I20260807 18:38:03.857736 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:03.858073 6697 external_mini_cluster.cc:1428] Setting key 83643a4e5734fdd33b68a47ffd7b0c5b
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:03.863468 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:03.850952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:32897 (local address 127.6.138.126:39299)
0807 18:38:03.851172 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:03.851177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:03.852092 (+ 915us) server_negotiation.cc:407] Connection header received
0807 18:38:03.853517 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:03.853520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:03.853569 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:03.853648 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:03.854798 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.855367 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.856623 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.856836 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.856923 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:03.859706 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:03.859710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:03.859712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:03.859739 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:03.861308 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.861939 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.861943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.861945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.862038 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.862387 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.862391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.862393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.862587 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:03.862699 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:03.863185 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 18:38:03.863303 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:38:03.864378 7555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:03.851429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39299 (local address 127.6.138.67:32897)
0807 18:38:03.851924 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:03.851972 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:03.853310 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:03.853818 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:03.853824 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:03.854172 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:03.854665 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:03.854679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.855510 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.855513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:03.856507 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:03.856517 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.856974 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.856977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:03.857109 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:03.857675 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:03.857699 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:03.859533 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:03.861432 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:03.861438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:03.861452 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:03.861828 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:03.862135 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:03.862139 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:03.862141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:03.862286 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:03.862699 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:03.862707 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:03.862949 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 18:38:03.863138 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 18:38:03.863454 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":126,"threads_started":1}
I20260807 18:38:03.865469 7553 heartbeater.cc:347] Connected to a master server at 127.6.138.126:39299
I20260807 18:38:03.865723 7553 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:03.866225 7553 heartbeater.cc:511] Master 127.6.138.126:39299 requested a full tablet report, sending...
I20260807 18:38:03.867254 6778 ts_manager.cc:194] Registered new tserver with Master: 66cd8d6a5c7748f4aef55bd87ac0b20c (127.6.138.67:44253)
I20260807 18:38:03.867784 6778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:32897
I20260807 18:38:03.872949 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:03.900929 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:03.891234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34760 (local address 127.6.138.65:35231)
0807 18:38:03.891556 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:03.891572 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:03.891598 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:38:03.891724 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:03.891732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:03.892023 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:03.892157 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:03.892872 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.893669 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.894489 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.894857 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.894944 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:03.897464 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:03.897475 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:03.897489 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:03.897514 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:03.899073 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.899532 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.899539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.899540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.899631 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.899882 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.899884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.899885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.900147 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:03.900448 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:03.900543 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 18:38:03.900713 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":78,"threads_started":1}
W20260807 18:38:03.902007 6954 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34760
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:38:03 dist-test-slave-2d19 krb5kdc[6709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:03.931090 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:03.921941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.6.138.65:35231)
0807 18:38:03.922201 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:03.922206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:03.922347 (+ 141us) server_negotiation.cc:407] Connection header received
0807 18:38:03.922483 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:03.922486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:03.922534 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:03.922628 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:03.923583 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.924120 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.925057 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:03.925285 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:03.925369 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:03.927958 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:03.927961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:03.927963 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:03.927988 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:03.929559 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.929935 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.929938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.929939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.930014 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:03.930262 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:03.930264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:03.930266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:03.930425 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:03.930545 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:03.930743 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 18:38:03.930844 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 18:38:03.932243 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 6890
I20260807 18:38:03.949223 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7098
I20260807 18:38:03.959523 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7324
I20260807 18:38:03.975440 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 6725
2026-08-07T18:38:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2655 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:04 dist-test-slave-2d19 krb5kdc[7586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:04 dist-test-slave-2d19 krb5kdc[7586](info): set up 2 sockets
Aug 07 18:38:04 dist-test-slave-2d19 krb5kdc[7586](info): commencing operation
krb5kdc: starting...
W20260807 18:38:05.234061 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:05 dist-test-slave-2d19 krb5kdc[7586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:05.366869 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:34535
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:34535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:05.484658 7609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:05.484929 7609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:05.484972 7609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:05.489095 7609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:05.489168 7609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:05.489187 7609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:05.489204 7609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:05.489241 7609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:05.494907 7609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:34535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:34535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:05.496088 7609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:05.497205 7609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:38:05.506256 7609 server_base.cc:1034] running on GCE node
W20260807 18:38:05.506212 7617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:05.506212 7622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:05.506186 7616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:05.507068 7609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:05.508181 7609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:05.509354 7609 hybrid_clock.cc:648] HybridClock initialized: now 1786127885509322 us; error 46 us; skew 500 ppm
Aug 07 18:38:05 dist-test-slave-2d19 krb5kdc[7586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127885, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:05.516985 7609 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:05.517946 7609 webserver.cc:460] Webserver started at http://127.6.138.126:35671/ using document root <none> and password file <none>
I20260807 18:38:05.518505 7609 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:05.518566 7609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:05.518739 7609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:05.520892 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "66d3dace23b743718690e703f94eeb91"
format_stamp: "Formatted at 2026-08-07 18:38:05 on dist-test-slave-2d19"
server_key: "f5099774dbf6f16f8e4e5bbc972cc8ea"
server_key_iv: "e1da86fd81d615776a7795feacf1ed27"
server_key_version: "encryptionkey@0"
I20260807 18:38:05.521404 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "66d3dace23b743718690e703f94eeb91"
format_stamp: "Formatted at 2026-08-07 18:38:05 on dist-test-slave-2d19"
server_key: "f5099774dbf6f16f8e4e5bbc972cc8ea"
server_key_iv: "e1da86fd81d615776a7795feacf1ed27"
server_key_version: "encryptionkey@0"
I20260807 18:38:05.525250 7609 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:38:05.527680 7630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:05.529175 7609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:05.529345 7609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "66d3dace23b743718690e703f94eeb91"
format_stamp: "Formatted at 2026-08-07 18:38:05 on dist-test-slave-2d19"
server_key: "f5099774dbf6f16f8e4e5bbc972cc8ea"
server_key_iv: "e1da86fd81d615776a7795feacf1ed27"
server_key_version: "encryptionkey@0"
I20260807 18:38:05.529492 7609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:05.554893 7609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:05.558481 7609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:05.558738 7609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:05.566450 7609 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:34535
I20260807 18:38:05.566498 7712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:34535 every 8 connection(s)
I20260807 18:38:05.567495 7609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:05.570525 7714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:05.575276 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7609
I20260807 18:38:05.575414 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:05.575688 6697 external_mini_cluster.cc:1428] Setting key df23bd5ef1dcdb45a4647196bd06e2c0
I20260807 18:38:05.576438 7714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91: Bootstrap starting.
I20260807 18:38:05.578902 7714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:05.579885 7714 log.cc:824] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:05.582110 7714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91: No bootstrap required, opened a new log
Aug 07 18:38:05 dist-test-slave-2d19 krb5kdc[7586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:05.586660 7714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d3dace23b743718690e703f94eeb91" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34535 } }
I20260807 18:38:05.586928 7714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:05.587025 7714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d3dace23b743718690e703f94eeb91, State: Initialized, Role: FOLLOWER
I20260807 18:38:05.587530 7714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d3dace23b743718690e703f94eeb91" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34535 } }
I20260807 18:38:05.587687 7714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:05.587778 7714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:05.587885 7714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:05.588901 7714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d3dace23b743718690e703f94eeb91" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34535 } }
I20260807 18:38:05.589288 7714 leader_election.cc:302] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66d3dace23b743718690e703f94eeb91; no voters:
I20260807 18:38:05.589637 7714 leader_election.cc:288] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:05.589758 7722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:05.589989 7722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [term 1 LEADER]: Becoming Leader. State: Replica: 66d3dace23b743718690e703f94eeb91, State: Running, Role: LEADER
I20260807 18:38:05.590296 7722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d3dace23b743718690e703f94eeb91" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34535 } }
I20260807 18:38:05.590818 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:05.577105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.6.138.126:34535)
0807 18:38:05.577512 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:05.577529 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:05.577567 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:38:05.578222 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:05.578240 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:05.578568 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:05.578848 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:05.579783 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:05.580882 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:05.582021 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:05.582527 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:05.582646 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:05.585465 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:05.585470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:05.585483 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:05.585510 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:05.587379 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:05.587871 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:05.587879 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:05.587885 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:05.588079 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:05.588420 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:05.588424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:05.588426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:05.588835 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:05.589067 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:05.589321 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 18:38:05.589816 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":126,"threads_started":1}
I20260807 18:38:05.590966 7714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:05.591529 7722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66d3dace23b743718690e703f94eeb91. Latest consensus state: current_term: 1 leader_uuid: "66d3dace23b743718690e703f94eeb91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d3dace23b743718690e703f94eeb91" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34535 } } }
I20260807 18:38:05.591639 7722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:05.592025 7724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66d3dace23b743718690e703f94eeb91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d3dace23b743718690e703f94eeb91" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34535 } } }
I20260807 18:38:05.592136 7724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91 [sys.catalog]: This master's current role is: LEADER
W20260807 18:38:05.596164 7748 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:38:05.596231 7748 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:38:05.596354 7727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:05.597268 7727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:05.602885 7727 catalog_manager.cc:1347] Generated new cluster ID: 9a74b114a6f4434387ba4ee63a0f183b
I20260807 18:38:05.602964 7727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:05.621388 7727 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:05.622627 7727 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:05.626895 7727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66d3dace23b743718690e703f94eeb91: Generated new TSK 0
I20260807 18:38:05.627612 7727 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:05.662858 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34535
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:05.812160 7757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:05.812444 7757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:05.812490 7757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:05.816463 7757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:05.816521 7757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:05.816583 7757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:05.821825 7757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34535
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:05.822890 7757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:05.823904 7757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:05.834177 7767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:05.834177 7769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:05.834232 7757 server_base.cc:1034] running on GCE node
W20260807 18:38:05.834177 7771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:05.834867 7757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:05.835525 7757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:05.836701 7757 hybrid_clock.cc:648] HybridClock initialized: now 1786127885836673 us; error 36 us; skew 500 ppm
Aug 07 18:38:05 dist-test-slave-2d19 krb5kdc[7586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127885, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:05.839753 7757 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:05.840752 7757 webserver.cc:460] Webserver started at http://127.6.138.65:41197/ using document root <none> and password file <none>
I20260807 18:38:05.841346 7757 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:05.841408 7757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:05.841634 7757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:05.843746 7757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "3599c418c6814471ab50bf44e77a9fc7"
format_stamp: "Formatted at 2026-08-07 18:38:05 on dist-test-slave-2d19"
server_key: "7ef0a32ee006759f1377b73151fe3ed6"
server_key_iv: "ec1af0c4027078840e3d71d4b4b74dec"
server_key_version: "encryptionkey@0"
I20260807 18:38:05.844395 7757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "3599c418c6814471ab50bf44e77a9fc7"
format_stamp: "Formatted at 2026-08-07 18:38:05 on dist-test-slave-2d19"
server_key: "7ef0a32ee006759f1377b73151fe3ed6"
server_key_iv: "ec1af0c4027078840e3d71d4b4b74dec"
server_key_version: "encryptionkey@0"
I20260807 18:38:05.848424 7757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:38:05.850893 7782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:05.852244 7757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:05.852447 7757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "3599c418c6814471ab50bf44e77a9fc7"
format_stamp: "Formatted at 2026-08-07 18:38:05 on dist-test-slave-2d19"
server_key: "7ef0a32ee006759f1377b73151fe3ed6"
server_key_iv: "ec1af0c4027078840e3d71d4b4b74dec"
server_key_version: "encryptionkey@0"
I20260807 18:38:05.852566 7757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:05.876614 7757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:05.879716 7757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:05.879935 7757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:05.880622 7757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:05.881538 7757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:05.881587 7757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:05.881659 7757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:05.881690 7757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:05.891508 7757 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:41637
I20260807 18:38:05.891563 7983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:41637 every 8 connection(s)
I20260807 18:38:05.892637 7757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:05.902029 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7757
Aug 07 18:38:05 dist-test-slave-2d19 krb5kdc[7586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127885, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:05.902179 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:05.902451 6697 external_mini_cluster.cc:1428] Setting key 54da8904ca2c5fb5395d9d1b7bd414fc
I20260807 18:38:05.907045 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:05.894553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:58967 (local address 127.6.138.126:34535)
0807 18:38:05.894713 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:05.894716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:05.895526 (+ 810us) server_negotiation.cc:407] Connection header received
0807 18:38:05.896874 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:05.896877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:05.896926 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:05.897010 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:05.898181 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:05.898712 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:05.899826 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:05.900062 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:05.900141 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:05.903000 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:05.903023 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:05.903028 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:05.903062 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:05.904728 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:05.905396 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:05.905398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:05.905400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:05.905475 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:05.905878 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:05.905881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:05.905882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:05.906053 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:05.906208 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:05.906743 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 18:38:05.906879 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:38:05.908042 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:05.894891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:34535 (local address 127.6.138.65:58967)
0807 18:38:05.895360 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:05.895406 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:05.896674 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:05.897192 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:05.897198 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:05.897562 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:05.898037 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:05.898050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:05.898822 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:05.898825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:05.899716 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:05.899725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:05.900207 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:05.900210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:05.900375 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:05.901017 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:05.901034 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:05.902794 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:05.904858 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:05.904869 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:05.904889 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:05.905292 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:05.905570 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:05.905575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:05.905580 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:05.905779 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:05.906213 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:05.906223 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:05.906486 (+ 263us) client_negotiation.cc:769] Sending connection context
0807 18:38:05.906700 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 18:38:05.907042 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":116,"threads_started":1}
I20260807 18:38:05.909413 7984 heartbeater.cc:347] Connected to a master server at 127.6.138.126:34535
I20260807 18:38:05.909715 7984 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:05.910359 7984 heartbeater.cc:511] Master 127.6.138.126:34535 requested a full tablet report, sending...
I20260807 18:38:05.911974 7653 ts_manager.cc:194] Registered new tserver with Master: 3599c418c6814471ab50bf44e77a9fc7 (127.6.138.65:41637)
I20260807 18:38:05.913197 7653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:58967
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:05.935410 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34535
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:06.051342 7991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:06.051631 7991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:06.051679 7991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:06.055716 7991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:06.055778 7991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:06.055840 7991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:06.061112 7991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34535
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:06.062218 7991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:06.063284 7991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:06.073324 8003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:06.073324 7999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:06.073326 8006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:06.073364 7991 server_base.cc:1034] running on GCE node
I20260807 18:38:06.073999 7991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:06.074687 7991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:06.075960 7991 hybrid_clock.cc:648] HybridClock initialized: now 1786127886075898 us; error 28 us; skew 500 ppm
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[7586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:06.079015 7991 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:06.079923 7991 webserver.cc:460] Webserver started at http://127.6.138.66:43089/ using document root <none> and password file <none>
I20260807 18:38:06.080533 7991 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:06.080600 7991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:06.080768 7991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:06.082875 7991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "85042e6d63da4005a99597b301197544"
format_stamp: "Formatted at 2026-08-07 18:38:06 on dist-test-slave-2d19"
server_key: "e0b3be0ebadf9b1aa414c66940d1c786"
server_key_iv: "596febf5e00188f3d093241cca6092c0"
server_key_version: "encryptionkey@0"
I20260807 18:38:06.083467 7991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "85042e6d63da4005a99597b301197544"
format_stamp: "Formatted at 2026-08-07 18:38:06 on dist-test-slave-2d19"
server_key: "e0b3be0ebadf9b1aa414c66940d1c786"
server_key_iv: "596febf5e00188f3d093241cca6092c0"
server_key_version: "encryptionkey@0"
I20260807 18:38:06.087523 7991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:38:06.090365 8016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:06.091708 7991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:38:06.091842 7991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "85042e6d63da4005a99597b301197544"
format_stamp: "Formatted at 2026-08-07 18:38:06 on dist-test-slave-2d19"
server_key: "e0b3be0ebadf9b1aa414c66940d1c786"
server_key_iv: "596febf5e00188f3d093241cca6092c0"
server_key_version: "encryptionkey@0"
I20260807 18:38:06.091995 7991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:06.105867 7991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:06.109246 7991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:06.109404 7991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:06.109974 7991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:06.110782 7991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:06.110829 7991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:06.110868 7991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:06.110889 7991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:06.120261 7991 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:32769
I20260807 18:38:06.120323 8212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:32769 every 8 connection(s)
I20260807 18:38:06.121239 7991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:06.122692 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7991
I20260807 18:38:06.122823 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:06.123123 6697 external_mini_cluster.cc:1428] Setting key ca99942490f5b1308e3eec436afbedac
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[7586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:38:06.137217 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.123120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:38375 (local address 127.6.138.126:34535)
0807 18:38:06.123290 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.123296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.124328 (+ 1032us) server_negotiation.cc:407] Connection header received
0807 18:38:06.125849 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.125852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.125908 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.125983 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:06.127459 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.128017 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.129922 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.130184 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.130272 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.133211 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:06.133214 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:06.133216 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:06.133277 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:06.134881 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.135550 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.135556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.135561 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.135667 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.136019 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.136021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.136023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.136199 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:06.136333 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.136941 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.137092 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
I20260807 18:38:06.138111 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.123569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:34535 (local address 127.6.138.66:38375)
0807 18:38:06.124153 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:06.124212 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:06.125627 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:06.126163 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:06.126170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:06.126529 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:06.127284 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.127307 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.128164 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.128171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.129766 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.129782 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.130356 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.130362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.130595 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.131250 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:06.131277 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:06.133037 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:06.135057 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:06.135062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:06.135074 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:06.135400 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:06.135753 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:06.135755 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:06.135756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:06.135900 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:06.136368 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:06.136373 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:06.136671 (+ 298us) client_negotiation.cc:769] Sending connection context
0807 18:38:06.136870 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 18:38:06.137153 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":150,"threads_started":1}
I20260807 18:38:06.139421 8214 heartbeater.cc:347] Connected to a master server at 127.6.138.126:34535
I20260807 18:38:06.139693 8214 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:06.140220 8214 heartbeater.cc:511] Master 127.6.138.126:34535 requested a full tablet report, sending...
I20260807 18:38:06.141660 7653 ts_manager.cc:194] Registered new tserver with Master: 85042e6d63da4005a99597b301197544 (127.6.138.66:32769)
I20260807 18:38:06.142592 7653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:38375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:06.155575 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34535
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:06.278047 8219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:06.278362 8219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:06.278438 8219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:06.282565 8219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:06.282651 8219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:06.282761 8219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:06.288125 8219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34535
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:06.289266 8219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:06.290328 8219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:06.301133 8229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:06.301131 8236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:06.301175 8219 server_base.cc:1034] running on GCE node
W20260807 18:38:06.301133 8230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:06.301846 8219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:06.302626 8219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:06.303826 8219 hybrid_clock.cc:648] HybridClock initialized: now 1786127886303794 us; error 45 us; skew 500 ppm
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[7586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:06.306978 8219 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:06.307972 8219 webserver.cc:460] Webserver started at http://127.6.138.67:34551/ using document root <none> and password file <none>
I20260807 18:38:06.308652 8219 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:06.308751 8219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:06.309003 8219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:06.311129 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "f913ea43233e4475afd0ef4e0d2ac373"
format_stamp: "Formatted at 2026-08-07 18:38:06 on dist-test-slave-2d19"
server_key: "55eb02480979f937333a87950f7f8130"
server_key_iv: "e090891dd0436aff6716d031d0843aee"
server_key_version: "encryptionkey@0"
I20260807 18:38:06.311738 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "f913ea43233e4475afd0ef4e0d2ac373"
format_stamp: "Formatted at 2026-08-07 18:38:06 on dist-test-slave-2d19"
server_key: "55eb02480979f937333a87950f7f8130"
server_key_iv: "e090891dd0436aff6716d031d0843aee"
server_key_version: "encryptionkey@0"
I20260807 18:38:06.315655 8219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 18:38:06.318259 8249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:06.319572 8219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:06.319718 8219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "f913ea43233e4475afd0ef4e0d2ac373"
format_stamp: "Formatted at 2026-08-07 18:38:06 on dist-test-slave-2d19"
server_key: "55eb02480979f937333a87950f7f8130"
server_key_iv: "e090891dd0436aff6716d031d0843aee"
server_key_version: "encryptionkey@0"
I20260807 18:38:06.319839 8219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:06.334772 8219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:06.338183 8219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:06.338411 8219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:06.339067 8219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:06.339960 8219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:06.340034 8219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:06.340113 8219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:06.340165 8219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:06.350356 8438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:35101 every 8 connection(s)
I20260807 18:38:06.350373 8219 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:35101
I20260807 18:38:06.351475 8219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:06.353190 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 8219
I20260807 18:38:06.353333 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:06.353699 6697 external_mini_cluster.cc:1428] Setting key 7fc128622353d31d1910adbf2555ab1a
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[7586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:06.366990 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.353773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:51627 (local address 127.6.138.126:34535)
0807 18:38:06.353944 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.353949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.355047 (+ 1098us) server_negotiation.cc:407] Connection header received
0807 18:38:06.356767 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.356769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.356816 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.356879 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:06.358061 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.358619 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.359753 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.359993 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.360085 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.363003 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:06.363007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:06.363009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:06.363041 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:06.364665 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.365255 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.365259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.365260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.365341 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.365716 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.365720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.365722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.365920 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:06.366041 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.366682 (+ 641us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.366822 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:38:06.368208 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.354307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:34535 (local address 127.6.138.67:51627)
0807 18:38:06.354853 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:06.354924 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:06.356514 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:06.357026 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:06.357033 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:06.357405 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:06.357917 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.357930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.358735 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.358737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.359632 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.359641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.360141 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.360144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.360358 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.361190 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:06.361207 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:06.362805 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:06.364799 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:06.364805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:06.364823 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:06.365130 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:06.365439 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:06.365443 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:06.365445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:06.365614 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:06.366035 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:06.366042 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:06.366386 (+ 344us) client_negotiation.cc:769] Sending connection context
0807 18:38:06.366644 (+ 258us) client_negotiation.cc:240] Negotiation successful
0807 18:38:06.367016 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":144,"threads_started":1}
I20260807 18:38:06.369419 8439 heartbeater.cc:347] Connected to a master server at 127.6.138.126:34535
I20260807 18:38:06.369681 8439 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:06.370234 8439 heartbeater.cc:511] Master 127.6.138.126:34535 requested a full tablet report, sending...
I20260807 18:38:06.371299 7653 ts_manager.cc:194] Registered new tserver with Master: f913ea43233e4475afd0ef4e0d2ac373 (127.6.138.67:35101)
I20260807 18:38:06.371905 7653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:51627
I20260807 18:38:06.380066 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:38:06.392125 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.383763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37796 (local address 127.6.138.126:34535)
0807 18:38:06.383946 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.383949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.384175 (+ 226us) server_negotiation.cc:407] Connection header received
0807 18:38:06.384318 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.384323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.384377 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.384468 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:06.385198 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.385734 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.386539 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.386764 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.386851 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.388405 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:06.388409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:06.388411 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:06.388438 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:06.390137 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.390572 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.390575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.390576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.390676 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.391061 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.391064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.391065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.391314 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:06.391438 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.391777 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.391976 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:38:06.397751 7653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:06.401180 7653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:06.417307 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.410687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.6.138.67:35101)
0807 18:38:06.412056 (+ 1369us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.412067 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.412089 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:38:06.412170 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.412176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.412381 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.412526 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:06.412763 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.413996 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.415647 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.416537 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.416658 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.416837 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.417013 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.417125 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1265,"thread_start_us":156,"threads_started":1}
I20260807 18:38:06.417301 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.410549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:32769 (local address 127.0.0.1:46864)
0807 18:38:06.411695 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:06.411712 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:06.411821 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:06.412402 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:06.412408 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:06.412436 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:06.412593 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.412614 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.414273 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.414276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.415739 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.415751 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.416872 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.416876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.416994 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.417019 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:38:06.417073 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 18:38:06.417163 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"mutex_wait_us":100,"thread_start_us":111,"threads_started":1}
I20260807 18:38:06.417678 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.411349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.6.138.65:41637)
0807 18:38:06.412108 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.412115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.412371 (+ 256us) server_negotiation.cc:407] Connection header received
0807 18:38:06.412510 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.412516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.412726 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.412916 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:06.413102 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.413817 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.415996 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.416675 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.416784 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.416934 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.417400 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.417531 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":65,"threads_started":1}
I20260807 18:38:06.418041 8463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.411250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46864 (local address 127.6.138.66:32769)
0807 18:38:06.411662 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.411676 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.411823 (+ 147us) server_negotiation.cc:407] Connection header received
0807 18:38:06.411964 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.411970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.412194 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.412777 (+ 583us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:06.412966 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.414062 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.415885 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.416730 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.417560 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.417715 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.417771 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.417879 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":149,"threads_started":1}
I20260807 18:38:06.417335 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.410511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:35101 (local address 127.0.0.1:57948)
0807 18:38:06.411519 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:06.411574 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:06.411713 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:06.412513 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:06.412516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:06.412531 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:06.412642 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.412648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.414139 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.414142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.415515 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.415525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.416689 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.416693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.416809 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.416847 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:38:06.416970 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 18:38:06.417102 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":114,"thread_start_us":81,"threads_started":1}
I20260807 18:38:06.419042 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.410891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:41637 (local address 127.0.0.1:48168)
0807 18:38:06.412282 (+ 1391us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:06.412296 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:06.412401 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:06.412876 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:06.412878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:06.412897 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:06.412998 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.413004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.413943 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.413948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.415873 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.415881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.416998 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.417001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.417114 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.417167 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 18:38:06.417241 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:38:06.417317 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"mutex_wait_us":339,"thread_start_us":105,"threads_started":1}
I20260807 18:38:06.421093 8096 tablet_service.cc:1467] Processing CreateTablet for tablet 19c73c231b1d4354bb5cac2886e3a444 (DEFAULT_TABLE table=test-table [id=96b6a06c969f4c3583a85111de4f2108]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:06.421160 8326 tablet_service.cc:1467] Processing CreateTablet for tablet 19c73c231b1d4354bb5cac2886e3a444 (DEFAULT_TABLE table=test-table [id=96b6a06c969f4c3583a85111de4f2108]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:06.421854 7866 tablet_service.cc:1467] Processing CreateTablet for tablet 19c73c231b1d4354bb5cac2886e3a444 (DEFAULT_TABLE table=test-table [id=96b6a06c969f4c3583a85111de4f2108]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:06.422264 8096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c73c231b1d4354bb5cac2886e3a444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:06.422339 8326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c73c231b1d4354bb5cac2886e3a444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:06.422885 7866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c73c231b1d4354bb5cac2886e3a444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:06.428884 8466 tablet_bootstrap.cc:492] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544: Bootstrap starting.
I20260807 18:38:06.430406 8467 tablet_bootstrap.cc:492] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7: Bootstrap starting.
I20260807 18:38:06.432511 8466 tablet_bootstrap.cc:654] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:06.432631 8468 tablet_bootstrap.cc:492] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373: Bootstrap starting.
I20260807 18:38:06.432749 8467 tablet_bootstrap.cc:654] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:06.433501 8466 log.cc:824] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:06.433585 8467 log.cc:824] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:06.435832 8468 tablet_bootstrap.cc:654] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:06.435837 8467 tablet_bootstrap.cc:492] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7: No bootstrap required, opened a new log
I20260807 18:38:06.435837 8466 tablet_bootstrap.cc:492] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544: No bootstrap required, opened a new log
I20260807 18:38:06.436057 8467 ts_tablet_manager.cc:1397] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:38:06.436057 8466 ts_tablet_manager.cc:1397] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 18:38:06.436694 8468 log.cc:824] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:06.438421 8468 tablet_bootstrap.cc:492] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373: No bootstrap required, opened a new log
I20260807 18:38:06.438642 8468 ts_tablet_manager.cc:1397] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:38:06.440423 8466 raft_consensus.cc:348] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } }
I20260807 18:38:06.440753 8466 raft_consensus.cc:374] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:06.440814 8466 raft_consensus.cc:729] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85042e6d63da4005a99597b301197544, State: Initialized, Role: FOLLOWER
I20260807 18:38:06.440732 8467 raft_consensus.cc:348] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } }
I20260807 18:38:06.440994 8467 raft_consensus.cc:374] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:06.441067 8467 raft_consensus.cc:729] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3599c418c6814471ab50bf44e77a9fc7, State: Initialized, Role: FOLLOWER
I20260807 18:38:06.441345 8466 consensus_queue.cc:260] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } }
I20260807 18:38:06.441514 8467 consensus_queue.cc:260] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } }
I20260807 18:38:06.442221 7984 heartbeater.cc:503] Master 127.6.138.126:34535 was elected leader, sending a full tablet report...
I20260807 18:38:06.442489 8466 ts_tablet_manager.cc:1428] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:38:06.442579 8468 raft_consensus.cc:348] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } }
I20260807 18:38:06.442878 8468 raft_consensus.cc:374] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:06.442881 8214 heartbeater.cc:503] Master 127.6.138.126:34535 was elected leader, sending a full tablet report...
I20260807 18:38:06.442945 8468 raft_consensus.cc:729] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f913ea43233e4475afd0ef4e0d2ac373, State: Initialized, Role: FOLLOWER
I20260807 18:38:06.443387 8467 ts_tablet_manager.cc:1428] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 18:38:06.443396 8468 consensus_queue.cc:260] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } }
I20260807 18:38:06.444242 8439 heartbeater.cc:503] Master 127.6.138.126:34535 was elected leader, sending a full tablet report...
I20260807 18:38:06.444386 8468 ts_tablet_manager.cc:1428] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260807 18:38:06.602756 8441 tablet.cc:2369] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:38:06.622434 8215 tablet.cc:2369] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:38:06.644170 7985 tablet.cc:2369] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:06.703466 8475 raft_consensus.cc:482] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:06.703766 8475 raft_consensus.cc:504] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } }
I20260807 18:38:06.705107 8475 leader_election.cc:288] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f913ea43233e4475afd0ef4e0d2ac373 (127.6.138.67:35101), 85042e6d63da4005a99597b301197544 (127.6.138.66:32769)
I20260807 18:38:06.710132 8479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.705417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:32769 (local address 127.6.138.65:44783)
0807 18:38:06.705893 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:06.705909 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:06.706059 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:06.706397 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:06.706400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:06.706425 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:06.706665 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.706672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.707774 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.707777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.708907 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.708918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.709615 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.709622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.709728 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.709786 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 18:38:06.709861 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 18:38:06.709924 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":106,"spinlock_wait_cycles":24320,"thread_start_us":96,"threads_started":1}
I20260807 18:38:06.710131 8463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.705629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:44783 (local address 127.6.138.66:32769)
0807 18:38:06.705755 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.705762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.706121 (+ 359us) server_negotiation.cc:407] Connection header received
0807 18:38:06.706198 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.706201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.706262 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.706373 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:06.706832 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.707651 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.709034 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.709478 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.709584 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.709626 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.709908 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.710007 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:38:06.710237 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.705537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:50131 (local address 127.6.138.67:35101)
0807 18:38:06.705693 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.705698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.706127 (+ 429us) server_negotiation.cc:407] Connection header received
0807 18:38:06.706199 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.706202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.706262 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.706373 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:06.706832 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.707651 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.709148 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.709582 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.709717 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.709776 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.710018 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.710099 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:38:06.710582 8478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.705417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:35101 (local address 127.6.138.65:50131)
0807 18:38:06.705872 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:06.705888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:06.706059 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:06.706397 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:06.706400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:06.706425 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:06.706665 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.706672 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.707850 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.707854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.709007 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:06.709020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.709729 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.709732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:06.709866 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.709893 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:38:06.709971 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 18:38:06.710038 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":36,"spinlock_wait_cycles":18816,"thread_start_us":130,"threads_started":1}
I20260807 18:38:06.711035 8127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c73c231b1d4354bb5cac2886e3a444" candidate_uuid: "3599c418c6814471ab50bf44e77a9fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85042e6d63da4005a99597b301197544" is_pre_election: true
I20260807 18:38:06.711272 8370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c73c231b1d4354bb5cac2886e3a444" candidate_uuid: "3599c418c6814471ab50bf44e77a9fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f913ea43233e4475afd0ef4e0d2ac373" is_pre_election: true
I20260807 18:38:06.711381 8127 raft_consensus.cc:2454] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3599c418c6814471ab50bf44e77a9fc7 in term 0.
I20260807 18:38:06.711612 8370 raft_consensus.cc:2454] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3599c418c6814471ab50bf44e77a9fc7 in term 0.
I20260807 18:38:06.711956 7784 leader_election.cc:302] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3599c418c6814471ab50bf44e77a9fc7, 85042e6d63da4005a99597b301197544; no voters:
I20260807 18:38:06.712308 8475 raft_consensus.cc:2781] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:06.712397 8475 raft_consensus.cc:482] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:06.712428 8475 raft_consensus.cc:3037] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:06.713523 8475 raft_consensus.cc:504] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } }
I20260807 18:38:06.713922 8475 leader_election.cc:288] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [CANDIDATE]: Term 1 election: Requested vote from peers f913ea43233e4475afd0ef4e0d2ac373 (127.6.138.67:35101), 85042e6d63da4005a99597b301197544 (127.6.138.66:32769)
I20260807 18:38:06.714254 8370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c73c231b1d4354bb5cac2886e3a444" candidate_uuid: "3599c418c6814471ab50bf44e77a9fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f913ea43233e4475afd0ef4e0d2ac373"
I20260807 18:38:06.714380 8127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c73c231b1d4354bb5cac2886e3a444" candidate_uuid: "3599c418c6814471ab50bf44e77a9fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85042e6d63da4005a99597b301197544"
I20260807 18:38:06.714375 8370 raft_consensus.cc:3037] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:06.714538 8127 raft_consensus.cc:3037] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:06.715579 8370 raft_consensus.cc:2454] T 19c73c231b1d4354bb5cac2886e3a444 P f913ea43233e4475afd0ef4e0d2ac373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3599c418c6814471ab50bf44e77a9fc7 in term 1.
I20260807 18:38:06.715620 8127 raft_consensus.cc:2454] T 19c73c231b1d4354bb5cac2886e3a444 P 85042e6d63da4005a99597b301197544 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3599c418c6814471ab50bf44e77a9fc7 in term 1.
I20260807 18:38:06.715927 7786 leader_election.cc:302] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3599c418c6814471ab50bf44e77a9fc7, f913ea43233e4475afd0ef4e0d2ac373; no voters:
I20260807 18:38:06.716131 8475 raft_consensus.cc:2781] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:06.716416 8475 raft_consensus.cc:686] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [term 1 LEADER]: Becoming Leader. State: Replica: 3599c418c6814471ab50bf44e77a9fc7, State: Running, Role: LEADER
I20260807 18:38:06.716707 8475 consensus_queue.cc:237] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } }
I20260807 18:38:06.719974 7653 catalog_manager.cc:5495] T 19c73c231b1d4354bb5cac2886e3a444 P 3599c418c6814471ab50bf44e77a9fc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3599c418c6814471ab50bf44e77a9fc7 (127.6.138.65). New cstate: current_term: 1 leader_uuid: "3599c418c6814471ab50bf44e77a9fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f913ea43233e4475afd0ef4e0d2ac373" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 35101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85042e6d63da4005a99597b301197544" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 32769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3599c418c6814471ab50bf44e77a9fc7" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41637 } health_report { overall_health: HEALTHY } } }
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[7586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:06.790716 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.780981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48174 (local address 127.6.138.65:41637)
0807 18:38:06.781164 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.781169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.781299 (+ 130us) server_negotiation.cc:407] Connection header received
0807 18:38:06.781519 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.781521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.781569 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.781660 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:06.782384 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.782907 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.783805 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.784034 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.784141 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.786802 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:06.786826 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:06.786843 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:06.786867 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:06.788812 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.789282 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.789288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.789290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.789386 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.789683 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.789687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.789689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.789978 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:06.790204 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.790410 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.790551 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[7586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127886, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[7586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127886, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:06.820134 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.811172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.6.138.65:41637)
0807 18:38:06.811344 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.811349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.811511 (+ 162us) server_negotiation.cc:407] Connection header received
0807 18:38:06.811688 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.811691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.811742 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.811832 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:06.812714 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.813229 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.814089 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.814314 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.814424 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.816889 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:06.816893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:06.816902 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:06.816928 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:06.818377 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.818784 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.818787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.818788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.818867 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.819129 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.819132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.819133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.819353 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:06.819516 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.819809 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.819963 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[7586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[7586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:06.848554 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:06.839704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.6.138.65:41637)
0807 18:38:06.839869 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:06.839874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:06.840000 (+ 126us) server_negotiation.cc:407] Connection header received
0807 18:38:06.840203 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:06.840205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:06.840294 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:06.840390 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:06.841111 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.841640 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.842529 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:06.842769 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:06.842883 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:06.845460 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:06.845468 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:06.845470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:06.845496 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:06.846748 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.847199 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.847203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.847205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.847295 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:06.847580 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:06.847584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:06.847586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:06.847774 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:06.847940 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:06.848197 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 18:38:06.848373 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 18:38:06.849972 7837 tablet_service.cc:2908] Rejecting scan request for tablet 19c73c231b1d4354bb5cac2886e3a444: Uninitialized: clean time has not yet been initialized
I20260807 18:38:06.852162 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7757
I20260807 18:38:06.875943 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7991
I20260807 18:38:06.887403 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 8219
I20260807 18:38:06.901871 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7609
2026-08-07T18:38:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2927 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[8504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[8504](info): set up 2 sockets
Aug 07 18:38:06 dist-test-slave-2d19 krb5kdc[8504](info): commencing operation
krb5kdc: starting...
W20260807 18:38:08.162245 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:08 dist-test-slave-2d19 krb5kdc[8504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:08.295881 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42131
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:08.414419 8524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:08.414737 8524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:08.414824 8524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:08.418895 8524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:08.418998 8524 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:38:08.419029 8524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:08.419071 8524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:08.419142 8524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:08.419181 8524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:08.424677 8524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42131
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:08.425881 8524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:08.426968 8524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:08.436237 8539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:08.436237 8534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:08.436446 8524 server_base.cc:1034] running on GCE node
W20260807 18:38:08.436288 8531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:08.437189 8524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:08.438355 8524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:08.439579 8524 hybrid_clock.cc:648] HybridClock initialized: now 1786127888439550 us; error 40 us; skew 500 ppm
Aug 07 18:38:08 dist-test-slave-2d19 krb5kdc[8504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127888, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:08.442840 8524 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:08.443872 8524 webserver.cc:460] Webserver started at http://127.6.138.126:39683/ using document root <none> and password file <none>
I20260807 18:38:08.444574 8524 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:08.444679 8524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:08.444933 8524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:08.447118 8524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "82d9e9de011f453ebd9a073eca49e780"
format_stamp: "Formatted at 2026-08-07 18:38:08 on dist-test-slave-2d19"
server_key: "5e5433739b0a7de1b48765118deaaddd"
server_key_iv: "4aee496f695317a5d31215a2b482a3ce"
server_key_version: "encryptionkey@0"
I20260807 18:38:08.447736 8524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "82d9e9de011f453ebd9a073eca49e780"
format_stamp: "Formatted at 2026-08-07 18:38:08 on dist-test-slave-2d19"
server_key: "5e5433739b0a7de1b48765118deaaddd"
server_key_iv: "4aee496f695317a5d31215a2b482a3ce"
server_key_version: "encryptionkey@0"
I20260807 18:38:08.451689 8524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260807 18:38:08.454384 8553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:08.455665 8524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:38:08.455823 8524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "82d9e9de011f453ebd9a073eca49e780"
format_stamp: "Formatted at 2026-08-07 18:38:08 on dist-test-slave-2d19"
server_key: "5e5433739b0a7de1b48765118deaaddd"
server_key_iv: "4aee496f695317a5d31215a2b482a3ce"
server_key_version: "encryptionkey@0"
I20260807 18:38:08.455941 8524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:08.475394 8524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:08.478833 8524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:08.479054 8524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:08.487839 8524 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:42131
I20260807 18:38:08.487870 8655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:42131 every 8 connection(s)
I20260807 18:38:08.488865 8524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:08.491851 8656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:08.493863 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 8524
I20260807 18:38:08.493973 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:08.494261 6697 external_mini_cluster.cc:1428] Setting key 747e1959b12057cb9ead4f3ba7c087f7
I20260807 18:38:08.499053 8656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780: Bootstrap starting.
I20260807 18:38:08.501936 8656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:08.503046 8656 log.cc:824] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:38:08 dist-test-slave-2d19 krb5kdc[8504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:08.505343 8656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780: No bootstrap required, opened a new log
I20260807 18:38:08.508939 8656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82d9e9de011f453ebd9a073eca49e780" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42131 } }
I20260807 18:38:08.509212 8656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:08.509305 8656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82d9e9de011f453ebd9a073eca49e780, State: Initialized, Role: FOLLOWER
I20260807 18:38:08.509712 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:08.495889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36244 (local address 127.6.138.126:42131)
0807 18:38:08.496353 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:08.496379 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:08.496419 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:38:08.497090 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:08.497108 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:08.497447 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:08.497737 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:08.498705 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.499824 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.501071 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.501536 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.501663 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:08.504786 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:08.504792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:08.504806 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:08.504834 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:08.506637 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:08.507113 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:08.507126 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:08.507131 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:08.507244 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:08.507545 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:08.507548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:08.507552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:08.507951 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:08.508180 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:08.508480 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:38:08.508783 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":160,"threads_started":1}
I20260807 18:38:08.509789 8656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82d9e9de011f453ebd9a073eca49e780" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42131 } }
I20260807 18:38:08.509994 8656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:08.510083 8656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:08.510188 8656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:08.511386 8656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82d9e9de011f453ebd9a073eca49e780" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42131 } }
I20260807 18:38:08.511773 8656 leader_election.cc:302] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82d9e9de011f453ebd9a073eca49e780; no voters:
I20260807 18:38:08.512164 8656 leader_election.cc:288] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:08.512583 8666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:08.512782 8666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [term 1 LEADER]: Becoming Leader. State: Replica: 82d9e9de011f453ebd9a073eca49e780, State: Running, Role: LEADER
I20260807 18:38:08.513090 8666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82d9e9de011f453ebd9a073eca49e780" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42131 } }
I20260807 18:38:08.513511 8656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:08.514715 8670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82d9e9de011f453ebd9a073eca49e780. Latest consensus state: current_term: 1 leader_uuid: "82d9e9de011f453ebd9a073eca49e780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82d9e9de011f453ebd9a073eca49e780" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42131 } } }
I20260807 18:38:08.514851 8670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:08.515187 8668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82d9e9de011f453ebd9a073eca49e780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82d9e9de011f453ebd9a073eca49e780" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42131 } } }
I20260807 18:38:08.515280 8668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:08.515369 8683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:08.518931 8683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:08.524513 8683 catalog_manager.cc:1347] Generated new cluster ID: a1738e53d111485cb9dc076cbfc0b617
I20260807 18:38:08.524596 8683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:08.537137 8683 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:08.538167 8683 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:08.545442 8683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780: Generated new TSK 0
I20260807 18:38:08.546212 8683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:08.581743 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42131
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:08.703056 8702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:08.703325 8702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:08.703370 8702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:08.707381 8702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:08.707453 8702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:08.707527 8702 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:38:08.707612 8702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:08.713018 8702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.138.126:42131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:08.714296 8702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:08.715466 8702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:08.725457 8714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:08.725466 8711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:08.725531 8712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:08.725936 8702 server_base.cc:1034] running on GCE node
I20260807 18:38:08.726337 8702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:08.726986 8702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:08.728150 8702 hybrid_clock.cc:648] HybridClock initialized: now 1786127888728134 us; error 27 us; skew 500 ppm
Aug 07 18:38:08 dist-test-slave-2d19 krb5kdc[8504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127888, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:08.731362 8702 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:08.732347 8702 webserver.cc:460] Webserver started at http://127.6.138.65:40893/ using document root <none> and password file <none>
I20260807 18:38:08.732945 8702 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:08.733011 8702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:08.733258 8702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:08.735361 8702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "e1d2dc2700f941be838bf5a35081c0e6"
format_stamp: "Formatted at 2026-08-07 18:38:08 on dist-test-slave-2d19"
server_key: "b45450a798a52760ca7c4ddebce89bab"
server_key_iv: "4c182d3ee8f94ae369e5920ac8bc5726"
server_key_version: "encryptionkey@0"
I20260807 18:38:08.735949 8702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "e1d2dc2700f941be838bf5a35081c0e6"
format_stamp: "Formatted at 2026-08-07 18:38:08 on dist-test-slave-2d19"
server_key: "b45450a798a52760ca7c4ddebce89bab"
server_key_iv: "4c182d3ee8f94ae369e5920ac8bc5726"
server_key_version: "encryptionkey@0"
I20260807 18:38:08.739809 8702 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:38:08.742523 8728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:08.743772 8702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:08.743912 8702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "e1d2dc2700f941be838bf5a35081c0e6"
format_stamp: "Formatted at 2026-08-07 18:38:08 on dist-test-slave-2d19"
server_key: "b45450a798a52760ca7c4ddebce89bab"
server_key_iv: "4c182d3ee8f94ae369e5920ac8bc5726"
server_key_version: "encryptionkey@0"
I20260807 18:38:08.744040 8702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:08.763854 8702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:08.767382 8702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:08.767608 8702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:08.769328 8702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:08.769407 8702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:08.769474 8702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:08.769508 8702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:38:08 dist-test-slave-2d19 krb5kdc[8504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127888, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:08.781880 8702 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:41611
I20260807 18:38:08.782909 8702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:08.783782 8918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:41611 every 8 connection(s)
I20260807 18:38:08.784571 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:08.770130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:49587 (local address 127.6.138.126:42131)
0807 18:38:08.770323 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:08.770330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:08.771254 (+ 924us) server_negotiation.cc:407] Connection header received
0807 18:38:08.772294 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:08.772298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:08.772363 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:08.772450 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:08.774262 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.775077 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.776579 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.777004 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.777141 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:08.780275 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:08.780282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:08.780286 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:08.780316 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:08.782161 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:08.782803 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:08.782807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:08.782809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:08.782903 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:08.783319 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:08.783324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:08.783326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:08.783557 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:08.783668 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:08.784316 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 18:38:08.784443 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:38:08.785409 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:08.770548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42131 (local address 127.6.138.65:49587)
0807 18:38:08.771051 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:08.771122 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:08.772033 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:08.772703 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:08.772717 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:08.773262 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:08.774005 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:08.774032 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.775245 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.775249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:08.776438 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:08.776449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.777206 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.777213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:08.777435 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:08.778096 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:08.778117 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:08.780023 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:08.782344 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:08.782351 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:08.782367 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:08.782690 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:08.783041 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:08.783044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:08.783046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:08.783211 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:08.783726 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:08.783732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:08.784036 (+ 304us) client_negotiation.cc:769] Sending connection context
0807 18:38:08.784202 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 18:38:08.784528 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":143,"threads_started":1}
I20260807 18:38:08.786773 8919 heartbeater.cc:347] Connected to a master server at 127.6.138.126:42131
I20260807 18:38:08.786937 8919 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:08.787465 8919 heartbeater.cc:511] Master 127.6.138.126:42131 requested a full tablet report, sending...
I20260807 18:38:08.789580 8576 ts_manager.cc:194] Registered new tserver with Master: e1d2dc2700f941be838bf5a35081c0e6 (127.6.138.65:41611)
I20260807 18:38:08.790220 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 8702
I20260807 18:38:08.790335 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:08.790633 6697 external_mini_cluster.cc:1428] Setting key 9e7e7a8db28f0d4ae05667f496c2b181
I20260807 18:38:08.791105 8576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:49587
I20260807 18:38:08.797843 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:08.789442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:53335 (local address 127.6.138.126:42131)
0807 18:38:08.789577 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:08.789582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:08.789868 (+ 286us) server_negotiation.cc:407] Connection header received
0807 18:38:08.789991 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:08.789994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:08.790039 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:08.790139 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:08.790617 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.791307 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.792408 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.792723 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.792835 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:08.794294 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:08.794300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:08.794305 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:08.794338 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:08.795975 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:08.796585 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:08.796588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:08.796589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:08.796667 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:08.797017 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:08.797019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:08.797020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:08.797202 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:08.797281 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:08.797634 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 18:38:08.797729 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:38:08.798175 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:08.789363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42131 (local address 127.6.138.65:53335)
0807 18:38:08.789779 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:08.789796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:08.789911 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:08.790153 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:08.790156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:08.790392 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:08.790514 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:08.790521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.791431 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.791434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:08.792274 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:08.792286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:08.792857 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:08.792863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:08.793000 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:08.793605 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:08.793625 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:08.794134 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:08.796111 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:08.796117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:08.796119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:08.796469 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:08.796770 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:08.796777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:08.796782 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:08.796912 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:08.797304 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:08.797311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:08.797440 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 18:38:08.797593 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 18:38:08.797806 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:08.821614 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42131
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:08.941175 8935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:08.941529 8935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:08.941617 8935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:08.945955 8935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:08.946043 8935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:08.946113 8935 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:38:08.946199 8935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:08.951735 8935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.138.126:42131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:08.952930 8935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:08.954022 8935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:08.963799 8942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:08.963878 8935 server_base.cc:1034] running on GCE node
W20260807 18:38:08.963785 8941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:08.963785 8945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:08.964524 8935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:08.965232 8935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:08.966435 8935 hybrid_clock.cc:648] HybridClock initialized: now 1786127888966397 us; error 44 us; skew 500 ppm
Aug 07 18:38:08 dist-test-slave-2d19 krb5kdc[8504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127888, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:08.969437 8935 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:08.970341 8935 webserver.cc:460] Webserver started at http://127.6.138.66:46261/ using document root <none> and password file <none>
I20260807 18:38:08.970868 8935 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:08.970929 8935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:08.971177 8935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:08.973321 8935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "a9b5eafe5e514bdba1f81683f6077740"
format_stamp: "Formatted at 2026-08-07 18:38:08 on dist-test-slave-2d19"
server_key: "8d5960adc26b3ea863b98f450148d67e"
server_key_iv: "ec539c9d57e19fb9f0f30d7fa8280f1f"
server_key_version: "encryptionkey@0"
I20260807 18:38:08.973841 8935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "a9b5eafe5e514bdba1f81683f6077740"
format_stamp: "Formatted at 2026-08-07 18:38:08 on dist-test-slave-2d19"
server_key: "8d5960adc26b3ea863b98f450148d67e"
server_key_iv: "ec539c9d57e19fb9f0f30d7fa8280f1f"
server_key_version: "encryptionkey@0"
I20260807 18:38:08.977967 8935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:08.980393 8959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:08.981626 8935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:08.981752 8935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "a9b5eafe5e514bdba1f81683f6077740"
format_stamp: "Formatted at 2026-08-07 18:38:08 on dist-test-slave-2d19"
server_key: "8d5960adc26b3ea863b98f450148d67e"
server_key_iv: "ec539c9d57e19fb9f0f30d7fa8280f1f"
server_key_version: "encryptionkey@0"
I20260807 18:38:08.981865 8935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:09.017059 8935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:09.020699 8935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:09.020924 8935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:09.022470 8935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:09.022552 8935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:09.022614 8935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:09.022668 8935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:38:09 dist-test-slave-2d19 krb5kdc[8504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127888, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:09.034520 8935 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:46571
I20260807 18:38:09.034659 9149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:46571 every 8 connection(s)
I20260807 18:38:09.035581 8935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:09.037459 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.023352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:55467 (local address 127.6.138.126:42131)
0807 18:38:09.023476 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.023480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.024674 (+ 1194us) server_negotiation.cc:407] Connection header received
0807 18:38:09.025635 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.025638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.025698 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.025805 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:09.027474 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.028410 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.029714 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.030071 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.030199 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.033390 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:09.033395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:09.033398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:09.033434 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:09.035151 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.035758 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.035762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.035764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.035853 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.036310 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.036314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.036316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.036558 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:09.036669 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.037221 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.037337 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:38:09.038300 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.023976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42131 (local address 127.6.138.66:55467)
0807 18:38:09.024522 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.024569 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.025424 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.026122 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.026130 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.026656 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:09.027323 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.027347 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.028546 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.028549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.029575 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.029585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.030261 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.030265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.030472 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.031154 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:09.031171 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:09.033189 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:09.035273 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.035280 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.035294 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.035658 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.035957 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.035960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.035962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.036146 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.036675 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:09.036680 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:09.036969 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.037135 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.037413 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":133,"threads_started":1}
I20260807 18:38:09.038971 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 8935
I20260807 18:38:09.039104 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:09.039426 6697 external_mini_cluster.cc:1428] Setting key a7734a87e84114824993a56f2b62fc54
I20260807 18:38:09.039686 9150 heartbeater.cc:347] Connected to a master server at 127.6.138.126:42131
I20260807 18:38:09.039868 9150 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:09.040509 9150 heartbeater.cc:511] Master 127.6.138.126:42131 requested a full tablet report, sending...
I20260807 18:38:09.041582 8576 ts_manager.cc:194] Registered new tserver with Master: a9b5eafe5e514bdba1f81683f6077740 (127.6.138.66:46571)
I20260807 18:38:09.042362 8576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:55467
I20260807 18:38:09.050559 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.042437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:33031 (local address 127.6.138.126:42131)
0807 18:38:09.042598 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.042612 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.042625 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:38:09.042665 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.042667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.042698 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.042763 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:09.043389 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.044119 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.045413 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.045848 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.045976 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.047201 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:09.047208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:09.047214 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:09.047248 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:09.048842 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.049287 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.049291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.049293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.049375 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.049661 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.049666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.049668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.049886 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:09.049997 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.050291 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.050398 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:38:09.050571 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.042109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42131 (local address 127.6.138.66:33031)
0807 18:38:09.042442 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.042459 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.042563 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.042808 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.042814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.043069 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:09.043248 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.043258 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.044292 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.044298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.045293 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.045302 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.045966 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.045969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.046045 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.046596 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:09.046614 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:09.047061 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:09.048962 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.048967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.048969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.049181 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.049470 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.049474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.049476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.049566 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.049990 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:09.049993 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:09.050121 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.050232 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.050389 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:09.071097 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42131
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:09.193845 9163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:09.194113 9163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:09.194159 9163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:09.198282 9163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:09.198341 9163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:09.198380 9163 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:38:09.198441 9163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:09.203724 9163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.138.126:42131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:09.204864 9163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:09.205893 9163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:09.215832 9173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:09.215814 9175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:09.216123 9177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:09.216477 9163 server_base.cc:1034] running on GCE node
I20260807 18:38:09.216897 9163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:09.217533 9163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:09.218734 9163 hybrid_clock.cc:648] HybridClock initialized: now 1786127889218701 us; error 55 us; skew 500 ppm
Aug 07 18:38:09 dist-test-slave-2d19 krb5kdc[8504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127889, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:09.221732 9163 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:09.222694 9163 webserver.cc:460] Webserver started at http://127.6.138.67:38051/ using document root <none> and password file <none>
I20260807 18:38:09.223227 9163 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:09.223285 9163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:09.223448 9163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:09.225986 9163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "af8ae62212364cb1b363ecf9df7bbc63"
format_stamp: "Formatted at 2026-08-07 18:38:09 on dist-test-slave-2d19"
server_key: "f13922a0b8a5897801b01af4284ca0e0"
server_key_iv: "ec145849ddb3037f2065df34750c3e2b"
server_key_version: "encryptionkey@0"
I20260807 18:38:09.226675 9163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "af8ae62212364cb1b363ecf9df7bbc63"
format_stamp: "Formatted at 2026-08-07 18:38:09 on dist-test-slave-2d19"
server_key: "f13922a0b8a5897801b01af4284ca0e0"
server_key_iv: "ec145849ddb3037f2065df34750c3e2b"
server_key_version: "encryptionkey@0"
I20260807 18:38:09.230849 9163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:09.235333 9185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:09.237381 9163 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:38:09.237516 9163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "af8ae62212364cb1b363ecf9df7bbc63"
format_stamp: "Formatted at 2026-08-07 18:38:09 on dist-test-slave-2d19"
server_key: "f13922a0b8a5897801b01af4284ca0e0"
server_key_iv: "ec145849ddb3037f2065df34750c3e2b"
server_key_version: "encryptionkey@0"
I20260807 18:38:09.237660 9163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:09.269809 9163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:09.273417 9163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:09.273675 9163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:09.275487 9163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:09.275556 9163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:09.275653 9163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:09.275686 9163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:09.287839 9163 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:42991
I20260807 18:38:09.287951 9364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:42991 every 8 connection(s)
I20260807 18:38:09.288892 9163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
Aug 07 18:38:09 dist-test-slave-2d19 krb5kdc[8504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127889, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:09.292547 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 9163
I20260807 18:38:09.292647 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:09.292927 6697 external_mini_cluster.cc:1428] Setting key db13088a928fa3522b9a30de02668aca
I20260807 18:38:09.297004 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.279329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:44211 (local address 127.6.138.126:42131)
0807 18:38:09.279583 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.279591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.280213 (+ 622us) server_negotiation.cc:407] Connection header received
0807 18:38:09.283889 (+ 3676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.283896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.284011 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.284092 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:09.286388 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.287202 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.289118 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.289526 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.289655 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.292804 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:09.292807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:09.292809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:09.292836 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:09.294571 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.295167 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.295171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.295173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.295269 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.295680 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.295683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.295685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.295888 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:09.296021 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.296682 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.296815 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:38:09.297883 9201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.277871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42131 (local address 127.6.138.67:44211)
0807 18:38:09.279996 (+ 2125us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.280075 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.283506 (+ 3431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.284752 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.284762 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.285366 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:09.286098 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.286120 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.287363 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.287365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.288973 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.288988 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.289852 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.289857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.290086 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.290793 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:09.290812 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:09.292554 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:09.294715 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.294722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.294738 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.295040 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.295377 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.295380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.295383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.295559 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.296022 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:09.296030 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:09.296368 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.296616 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.296980 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1827,"thread_start_us":169,"threads_started":1}
I20260807 18:38:09.299309 9365 heartbeater.cc:347] Connected to a master server at 127.6.138.126:42131
I20260807 18:38:09.299494 9365 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:09.300104 9365 heartbeater.cc:511] Master 127.6.138.126:42131 requested a full tablet report, sending...
I20260807 18:38:09.301281 8576 ts_manager.cc:194] Registered new tserver with Master: af8ae62212364cb1b363ecf9df7bbc63 (127.6.138.67:42991)
I20260807 18:38:09.301975 8576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:44211
I20260807 18:38:09.307103 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:38:09.309580 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.301810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42131 (local address 127.6.138.67:53099)
0807 18:38:09.302167 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.302182 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.302296 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.302576 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.302579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.302795 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:09.302899 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.302905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.303667 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.303670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.304216 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.304224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.304794 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.304798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.304893 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.305474 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:09.305495 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:09.305843 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:09.307913 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.307916 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.307917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.308127 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.308463 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.308465 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.308466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.308555 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.309056 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:09.309059 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:09.309158 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.309270 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.309419 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":144,"threads_started":1}
I20260807 18:38:09.309718 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.301878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:53099 (local address 127.6.138.126:42131)
0807 18:38:09.302036 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.302041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.302271 (+ 230us) server_negotiation.cc:407] Connection header received
0807 18:38:09.302397 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.302401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.302448 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.302535 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:09.303018 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.303523 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.304447 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.304669 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.304785 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.305950 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:09.305953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:09.305956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:09.305983 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:09.307801 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.308230 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.308236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.308241 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.308357 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.308646 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.308652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.308656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.308910 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:09.309022 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.309354 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.309494 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 18:38:09 dist-test-slave-2d19 krb5kdc[8504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127889, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:38:09 dist-test-slave-2d19 krb5kdc[8504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127889, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:09.335525 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.327542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.6.138.126:42131)
0807 18:38:09.327691 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.327695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.327708 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:38:09.327749 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.327752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.327791 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.327860 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:09.328585 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.329102 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.329887 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.330106 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.330181 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.332489 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:09.332492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:09.332494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:09.332519 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:09.333912 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.334337 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.334339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.334341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.334415 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.334665 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.334667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.334669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.334830 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:09.334931 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.335218 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.335316 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:38:09.338598 8576 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:09.341939 8576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:09.357678 9397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.351146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.6.138.65:41611)
0807 18:38:09.352471 (+ 1325us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.352482 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.352513 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:38:09.352603 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.352609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.352844 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.353017 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:09.353338 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.354510 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.356337 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.356928 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.357070 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.357239 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.357422 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.357530 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1212,"thread_start_us":121,"threads_started":1}
I20260807 18:38:09.357681 9395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.350989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:41611 (local address 127.0.0.1:43022)
0807 18:38:09.351933 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.351989 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.352137 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.353012 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.353019 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.353042 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:09.353199 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.353209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.354608 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.354611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.356210 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.356220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.357071 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.357075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.357197 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.357238 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.357384 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.357517 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":345,"thread_start_us":151,"threads_started":1}
I20260807 18:38:09.357950 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.351905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.6.138.67:42991)
0807 18:38:09.352703 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.352714 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.352742 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:38:09.352811 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.352816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.352991 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.353126 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:09.353568 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.354388 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.356237 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.356999 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.357114 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.357278 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.357705 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.357805 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":144,"threads_started":1}
I20260807 18:38:09.357852 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.351344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:46571 (local address 127.0.0.1:42872)
0807 18:38:09.352687 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.352701 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.352792 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.353474 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.353476 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.353514 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:09.353631 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.353638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.354980 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.354985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.355964 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.355974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.357346 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.357350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.357542 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.357559 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.357619 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.357713 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1279,"mutex_wait_us":44,"thread_start_us":92,"threads_started":1}
I20260807 18:38:09.358239 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.351719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.6.138.66:46571)
0807 18:38:09.352528 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.352541 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.352885 (+ 344us) server_negotiation.cc:407] Connection header received
0807 18:38:09.352990 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.352995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.353188 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.353337 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:09.353766 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.354878 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.356137 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.356999 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.357114 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.357279 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.357956 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.358061 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":130,"threads_started":1}
I20260807 18:38:09.358577 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.351216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:42991 (local address 127.0.0.1:60004)
0807 18:38:09.352152 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.352167 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.352247 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.353305 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.353308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.353328 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:09.353431 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.353438 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.354524 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.354527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.356118 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.356130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.357346 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.357350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.357517 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.357543 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.357620 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.358437 (+ 817us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":841,"thread_start_us":106,"threads_started":1}
I20260807 18:38:09.361600 8808 tablet_service.cc:1467] Processing CreateTablet for tablet be2c2eba460c486f8c5ee330a9b27a24 (DEFAULT_TABLE table=test-table [id=39f526194c0543b687c22a703a08eba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:09.361624 9041 tablet_service.cc:1467] Processing CreateTablet for tablet be2c2eba460c486f8c5ee330a9b27a24 (DEFAULT_TABLE table=test-table [id=39f526194c0543b687c22a703a08eba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:09.361616 9260 tablet_service.cc:1467] Processing CreateTablet for tablet be2c2eba460c486f8c5ee330a9b27a24 (DEFAULT_TABLE table=test-table [id=39f526194c0543b687c22a703a08eba2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:09.362705 9041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2c2eba460c486f8c5ee330a9b27a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:09.362709 9260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2c2eba460c486f8c5ee330a9b27a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:09.362715 8808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2c2eba460c486f8c5ee330a9b27a24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:09.369211 9402 tablet_bootstrap.cc:492] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: Bootstrap starting.
I20260807 18:38:09.371671 9402 tablet_bootstrap.cc:654] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:09.371688 9403 tablet_bootstrap.cc:492] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: Bootstrap starting.
I20260807 18:38:09.372545 9402 log.cc:824] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:09.372648 9404 tablet_bootstrap.cc:492] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: Bootstrap starting.
I20260807 18:38:09.373986 9403 tablet_bootstrap.cc:654] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:09.374672 9402 tablet_bootstrap.cc:492] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: No bootstrap required, opened a new log
I20260807 18:38:09.374794 9404 tablet_bootstrap.cc:654] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:09.374878 9403 log.cc:824] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:09.374919 9402 ts_tablet_manager.cc:1397] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:38:09.375664 9404 log.cc:824] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:09.377333 9403 tablet_bootstrap.cc:492] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: No bootstrap required, opened a new log
I20260807 18:38:09.377529 9403 ts_tablet_manager.cc:1397] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:38:09.377921 9404 tablet_bootstrap.cc:492] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: No bootstrap required, opened a new log
I20260807 18:38:09.378103 9404 ts_tablet_manager.cc:1397] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:38:09.378990 9402 raft_consensus.cc:348] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } }
I20260807 18:38:09.379241 9402 raft_consensus.cc:374] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:09.379295 9402 raft_consensus.cc:729] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d2dc2700f941be838bf5a35081c0e6, State: Initialized, Role: FOLLOWER
I20260807 18:38:09.379739 9402 consensus_queue.cc:260] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } }
I20260807 18:38:09.380844 9402 ts_tablet_manager.cc:1428] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:38:09.381201 8919 heartbeater.cc:503] Master 127.6.138.126:42131 was elected leader, sending a full tablet report...
I20260807 18:38:09.381597 9403 raft_consensus.cc:348] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } }
I20260807 18:38:09.381862 9403 raft_consensus.cc:374] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:09.381929 9403 raft_consensus.cc:729] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b5eafe5e514bdba1f81683f6077740, State: Initialized, Role: FOLLOWER
I20260807 18:38:09.382022 9404 raft_consensus.cc:348] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } }
I20260807 18:38:09.382258 9404 raft_consensus.cc:374] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:09.382335 9404 raft_consensus.cc:729] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af8ae62212364cb1b363ecf9df7bbc63, State: Initialized, Role: FOLLOWER
I20260807 18:38:09.382431 9403 consensus_queue.cc:260] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } }
I20260807 18:38:09.382748 9404 consensus_queue.cc:260] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } }
I20260807 18:38:09.383131 9150 heartbeater.cc:503] Master 127.6.138.126:42131 was elected leader, sending a full tablet report...
I20260807 18:38:09.383728 9404 ts_tablet_manager.cc:1428] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:38:09.384119 9365 heartbeater.cc:503] Master 127.6.138.126:42131 was elected leader, sending a full tablet report...
I20260807 18:38:09.384464 9403 ts_tablet_manager.cc:1428] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 18:38:09.395859 9413 raft_consensus.cc:482] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:09.396080 9413 raft_consensus.cc:504] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } }
I20260807 18:38:09.397269 9413 leader_election.cc:288] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1d2dc2700f941be838bf5a35081c0e6 (127.6.138.65:41611), af8ae62212364cb1b363ecf9df7bbc63 (127.6.138.67:42991)
I20260807 18:38:09.401953 9397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.397774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:48059 (local address 127.6.138.65:41611)
0807 18:38:09.397916 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.397921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.397935 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:38:09.397995 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.397998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.398097 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.398180 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:09.398513 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.399385 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.400754 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.401393 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.401526 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.401586 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.401791 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.401855 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:38:09.401973 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.397545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:41611 (local address 127.6.138.66:48059)
0807 18:38:09.397686 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.397704 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.397806 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.398214 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.398218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.398242 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:09.398389 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.398396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.399515 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.399519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.400624 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.400634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.401527 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.401530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.401661 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.401687 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.401770 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.401840 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 18:38:09.402593 8843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2c2eba460c486f8c5ee330a9b27a24" candidate_uuid: "a9b5eafe5e514bdba1f81683f6077740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d2dc2700f941be838bf5a35081c0e6" is_pre_election: true
I20260807 18:38:09.402930 8843 raft_consensus.cc:2454] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b5eafe5e514bdba1f81683f6077740 in term 0.
I20260807 18:38:09.403322 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.397545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:42991 (local address 127.6.138.66:32983)
0807 18:38:09.398023 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.398038 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.398159 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.398399 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.398401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.398416 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:09.399067 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.399078 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.400174 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.400177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.401378 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.401389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.402283 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.402287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.402953 (+ 666us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.402982 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.403070 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.403142 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":129,"thread_start_us":161,"threads_started":1}
I20260807 18:38:09.403393 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.397654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:32983 (local address 127.6.138.67:42991)
0807 18:38:09.397784 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.397789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.398166 (+ 377us) server_negotiation.cc:407] Connection header received
0807 18:38:09.398248 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.398251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.398298 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.398380 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:09.399256 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.400057 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.401518 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.402114 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.402249 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.402306 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.403144 (+ 838us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.403259 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:38:09.403512 8967 leader_election.cc:302] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b5eafe5e514bdba1f81683f6077740, e1d2dc2700f941be838bf5a35081c0e6; no voters:
I20260807 18:38:09.403851 9413 raft_consensus.cc:2781] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:09.403930 9413 raft_consensus.cc:482] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:09.403960 9413 raft_consensus.cc:3037] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:09.403988 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2c2eba460c486f8c5ee330a9b27a24" candidate_uuid: "a9b5eafe5e514bdba1f81683f6077740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af8ae62212364cb1b363ecf9df7bbc63" is_pre_election: true
I20260807 18:38:09.404366 9278 raft_consensus.cc:2454] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b5eafe5e514bdba1f81683f6077740 in term 0.
I20260807 18:38:09.405340 9413 raft_consensus.cc:504] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } }
I20260807 18:38:09.405707 9413 leader_election.cc:288] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [CANDIDATE]: Term 1 election: Requested vote from peers e1d2dc2700f941be838bf5a35081c0e6 (127.6.138.65:41611), af8ae62212364cb1b363ecf9df7bbc63 (127.6.138.67:42991)
I20260807 18:38:09.406116 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2c2eba460c486f8c5ee330a9b27a24" candidate_uuid: "a9b5eafe5e514bdba1f81683f6077740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af8ae62212364cb1b363ecf9df7bbc63"
I20260807 18:38:09.406131 8843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2c2eba460c486f8c5ee330a9b27a24" candidate_uuid: "a9b5eafe5e514bdba1f81683f6077740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d2dc2700f941be838bf5a35081c0e6"
I20260807 18:38:09.406256 9278 raft_consensus.cc:3037] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:09.406256 8843 raft_consensus.cc:3037] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:09.407637 9278 raft_consensus.cc:2454] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b5eafe5e514bdba1f81683f6077740 in term 1.
I20260807 18:38:09.407639 8843 raft_consensus.cc:2454] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b5eafe5e514bdba1f81683f6077740 in term 1.
I20260807 18:38:09.407959 8964 leader_election.cc:302] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b5eafe5e514bdba1f81683f6077740, af8ae62212364cb1b363ecf9df7bbc63; no voters:
I20260807 18:38:09.408171 9413 raft_consensus.cc:2781] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:09.408427 9413 raft_consensus.cc:686] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 1 LEADER]: Becoming Leader. State: Replica: a9b5eafe5e514bdba1f81683f6077740, State: Running, Role: LEADER
I20260807 18:38:09.408753 9413 consensus_queue.cc:237] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } }
I20260807 18:38:09.412806 8578 catalog_manager.cc:5495] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9b5eafe5e514bdba1f81683f6077740 (127.6.138.66). New cstate: current_term: 1 leader_uuid: "a9b5eafe5e514bdba1f81683f6077740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } health_report { overall_health: HEALTHY } } }
I20260807 18:38:09.443066 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.439099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42888 (local address 127.6.138.66:46571)
0807 18:38:09.439252 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.439259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.439276 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:38:09.439607 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.439610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.439673 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.439763 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:09.440031 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.440706 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.441623 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.441873 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.441977 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.442297 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:09.442728 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:09.442817 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.442897 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.442949 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:38:09.449540 8843 raft_consensus.cc:1261] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [term 1 FOLLOWER]: Refusing update from remote peer a9b5eafe5e514bdba1f81683f6077740: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:09.449855 9278 raft_consensus.cc:1261] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 FOLLOWER]: Refusing update from remote peer a9b5eafe5e514bdba1f81683f6077740: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:09.450395 9413 consensus_queue.cc:1035] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:09.450635 9421 consensus_queue.cc:1035] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:09.460968 9429 mvcc.cc:205] Tried to move back new op lower bound from 7315979835176669184 to 7315979835021139968. Current Snapshot: MvccSnapshot[applied={T|T < 7315979835176669184}]
I20260807 18:38:09.461905 9428 mvcc.cc:205] Tried to move back new op lower bound from 7315979835176669184 to 7315979835021139968. Current Snapshot: MvccSnapshot[applied={T|T < 7315979835176669184}]
I20260807 18:38:09.469599 8576 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:09.469765 8576 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:09.472420 8576 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780: Sending DeleteTablet for 3 replicas of tablet be2c2eba460c486f8c5ee330a9b27a24
I20260807 18:38:09.473274 9260 tablet_service.cc:1514] Processing DeleteTablet for tablet be2c2eba460c486f8c5ee330a9b27a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:09 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:60004
I20260807 18:38:09.473268 8808 tablet_service.cc:1514] Processing DeleteTablet for tablet be2c2eba460c486f8c5ee330a9b27a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:09 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:43022
I20260807 18:38:09.473464 9041 tablet_service.cc:1514] Processing DeleteTablet for tablet be2c2eba460c486f8c5ee330a9b27a24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:09 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:42872
I20260807 18:38:09.473868 9438 tablet_replica.cc:330] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: stopping tablet replica
I20260807 18:38:09.473965 9442 tablet_replica.cc:330] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: stopping tablet replica
I20260807 18:38:09.474275 9438 raft_consensus.cc:2229] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:09.474275 9442 raft_consensus.cc:2229] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:09.474355 8576 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:38:09.474706 9442 raft_consensus.cc:2258] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 18:38:09.474793 8576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:09.475067 9438 raft_consensus.cc:2258] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:09.476298 9440 tablet_replica.cc:330] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: stopping tablet replica
I20260807 18:38:09.476619 9440 raft_consensus.cc:2229] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:09.476933 9440 raft_consensus.cc:2258] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:09.477465 9438 ts_tablet_manager.cc:1905] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:09.477885 9442 ts_tablet_manager.cc:1905] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:09.480219 9260 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe4fc7cc27047208a22289dd9d71625 (DEFAULT_TABLE table=test-table [id=6c11e942da8c469e8506e51755325878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:09.480293 8808 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe4fc7cc27047208a22289dd9d71625 (DEFAULT_TABLE table=test-table [id=6c11e942da8c469e8506e51755325878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:09.480515 9260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe4fc7cc27047208a22289dd9d71625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:09.480540 8808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe4fc7cc27047208a22289dd9d71625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:09.480738 9442 ts_tablet_manager.cc:1918] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:09.480829 9442 log.cc:1196] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/be2c2eba460c486f8c5ee330a9b27a24
I20260807 18:38:09.481151 9442 ts_tablet_manager.cc:1939] T be2c2eba460c486f8c5ee330a9b27a24 P a9b5eafe5e514bdba1f81683f6077740: Deleting consensus metadata
I20260807 18:38:09.482862 9404 tablet_bootstrap.cc:492] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: Bootstrap starting.
I20260807 18:38:09.483078 9402 tablet_bootstrap.cc:492] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: Bootstrap starting.
I20260807 18:38:09.483372 8556 catalog_manager.cc:4841] TS a9b5eafe5e514bdba1f81683f6077740 (127.6.138.66:46571): tablet be2c2eba460c486f8c5ee330a9b27a24 (table test-table [id=39f526194c0543b687c22a703a08eba2]) successfully deleted
I20260807 18:38:09.483911 9404 tablet_bootstrap.cc:654] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:09.483896 9041 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe4fc7cc27047208a22289dd9d71625 (DEFAULT_TABLE table=test-table [id=6c11e942da8c469e8506e51755325878]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:09.484037 9402 tablet_bootstrap.cc:654] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:09.484166 9041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe4fc7cc27047208a22289dd9d71625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:09.484704 9440 ts_tablet_manager.cc:1905] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:09.486200 9404 tablet_bootstrap.cc:492] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: No bootstrap required, opened a new log
I20260807 18:38:09.486289 9404 ts_tablet_manager.cc:1397] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:38:09.486419 9402 tablet_bootstrap.cc:492] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: No bootstrap required, opened a new log
I20260807 18:38:09.486495 9402 ts_tablet_manager.cc:1397] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:38:09.486594 9403 tablet_bootstrap.cc:492] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: Bootstrap starting.
I20260807 18:38:09.486826 9404 raft_consensus.cc:348] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.486935 9404 raft_consensus.cc:374] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:09.486968 9404 raft_consensus.cc:729] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af8ae62212364cb1b363ecf9df7bbc63, State: Initialized, Role: FOLLOWER
I20260807 18:38:09.487102 9404 consensus_queue.cc:260] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.487157 9402 raft_consensus.cc:348] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.487251 9402 raft_consensus.cc:374] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:09.487282 9402 raft_consensus.cc:729] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d2dc2700f941be838bf5a35081c0e6, State: Initialized, Role: FOLLOWER
I20260807 18:38:09.487375 9404 ts_tablet_manager.cc:1428] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 18:38:09.487394 9402 consensus_queue.cc:260] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.487623 9403 tablet_bootstrap.cc:654] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:09.487692 9402 ts_tablet_manager.cc:1428] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 18:38:09.487766 9440 ts_tablet_manager.cc:1918] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:09.487864 9440 log.cc:1196] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/be2c2eba460c486f8c5ee330a9b27a24
I20260807 18:38:09.488193 9440 ts_tablet_manager.cc:1939] T be2c2eba460c486f8c5ee330a9b27a24 P e1d2dc2700f941be838bf5a35081c0e6: Deleting consensus metadata
I20260807 18:38:09.489862 9438 ts_tablet_manager.cc:1918] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:09.490015 9438 log.cc:1196] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/be2c2eba460c486f8c5ee330a9b27a24
I20260807 18:38:09.490402 9438 ts_tablet_manager.cc:1939] T be2c2eba460c486f8c5ee330a9b27a24 P af8ae62212364cb1b363ecf9df7bbc63: Deleting consensus metadata
I20260807 18:38:09.490921 8557 catalog_manager.cc:4841] TS e1d2dc2700f941be838bf5a35081c0e6 (127.6.138.65:41611): tablet be2c2eba460c486f8c5ee330a9b27a24 (table test-table [id=39f526194c0543b687c22a703a08eba2]) successfully deleted
I20260807 18:38:09.491616 8555 catalog_manager.cc:4841] TS af8ae62212364cb1b363ecf9df7bbc63 (127.6.138.67:42991): tablet be2c2eba460c486f8c5ee330a9b27a24 (table test-table [id=39f526194c0543b687c22a703a08eba2]) successfully deleted
I20260807 18:38:09.492741 9403 tablet_bootstrap.cc:492] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: No bootstrap required, opened a new log
I20260807 18:38:09.492841 9403 ts_tablet_manager.cc:1397] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 18:38:09.493319 9403 raft_consensus.cc:348] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.493451 9403 raft_consensus.cc:374] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:09.493491 9403 raft_consensus.cc:729] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b5eafe5e514bdba1f81683f6077740, State: Initialized, Role: FOLLOWER
I20260807 18:38:09.493619 9403 consensus_queue.cc:260] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.493871 9403 ts_tablet_manager.cc:1428] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260807 18:38:09.534441 8921 tablet.cc:2369] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:38:09.536996 9151 tablet.cc:2369] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:38:09.540184 9366 tablet.cc:2369] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:09.669440 9413 raft_consensus.cc:482] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:09.669624 9413 raft_consensus.cc:504] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.670300 9413 leader_election.cc:288] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1d2dc2700f941be838bf5a35081c0e6 (127.6.138.65:41611), af8ae62212364cb1b363ecf9df7bbc63 (127.6.138.67:42991)
I20260807 18:38:09.670950 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe4fc7cc27047208a22289dd9d71625" candidate_uuid: "a9b5eafe5e514bdba1f81683f6077740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af8ae62212364cb1b363ecf9df7bbc63" is_pre_election: true
I20260807 18:38:09.670950 8843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe4fc7cc27047208a22289dd9d71625" candidate_uuid: "a9b5eafe5e514bdba1f81683f6077740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d2dc2700f941be838bf5a35081c0e6" is_pre_election: true
I20260807 18:38:09.671147 8843 raft_consensus.cc:2454] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b5eafe5e514bdba1f81683f6077740 in term 0.
I20260807 18:38:09.671146 9278 raft_consensus.cc:2454] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b5eafe5e514bdba1f81683f6077740 in term 0.
I20260807 18:38:09.671545 8967 leader_election.cc:302] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a9b5eafe5e514bdba1f81683f6077740, af8ae62212364cb1b363ecf9df7bbc63, e1d2dc2700f941be838bf5a35081c0e6; no voters:
I20260807 18:38:09.671765 9413 raft_consensus.cc:2781] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:09.671841 9413 raft_consensus.cc:482] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:09.671927 9413 raft_consensus.cc:3037] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:09.673171 9413 raft_consensus.cc:504] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.673615 9413 leader_election.cc:288] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [CANDIDATE]: Term 1 election: Requested vote from peers e1d2dc2700f941be838bf5a35081c0e6 (127.6.138.65:41611), af8ae62212364cb1b363ecf9df7bbc63 (127.6.138.67:42991)
I20260807 18:38:09.674109 8843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe4fc7cc27047208a22289dd9d71625" candidate_uuid: "a9b5eafe5e514bdba1f81683f6077740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d2dc2700f941be838bf5a35081c0e6"
I20260807 18:38:09.674126 9278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe4fc7cc27047208a22289dd9d71625" candidate_uuid: "a9b5eafe5e514bdba1f81683f6077740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af8ae62212364cb1b363ecf9df7bbc63"
I20260807 18:38:09.674263 8843 raft_consensus.cc:3037] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:09.674290 9278 raft_consensus.cc:3037] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:09.675273 8843 raft_consensus.cc:2454] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b5eafe5e514bdba1f81683f6077740 in term 1.
I20260807 18:38:09.675275 9278 raft_consensus.cc:2454] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b5eafe5e514bdba1f81683f6077740 in term 1.
I20260807 18:38:09.675627 8967 leader_election.cc:302] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b5eafe5e514bdba1f81683f6077740, e1d2dc2700f941be838bf5a35081c0e6; no voters:
I20260807 18:38:09.675833 9413 raft_consensus.cc:2781] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:09.676017 9413 raft_consensus.cc:686] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 1 LEADER]: Becoming Leader. State: Replica: a9b5eafe5e514bdba1f81683f6077740, State: Running, Role: LEADER
I20260807 18:38:09.676182 9413 consensus_queue.cc:237] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.678098 8576 catalog_manager.cc:5495] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9b5eafe5e514bdba1f81683f6077740 (127.6.138.66). New cstate: current_term: 1 leader_uuid: "a9b5eafe5e514bdba1f81683f6077740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:38:09 dist-test-slave-2d19 krb5kdc[8504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127888, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:09.768146 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.750189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42131 (local address 127.0.0.1:36264)
0807 18:38:09.750511 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:09.750541 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:09.750650 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:09.750888 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:09.750890 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:09.751307 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:09.751411 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.751417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.752420 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.752422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.752987 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:09.752995 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.753452 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.753454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:09.753515 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.762613 (+ 9098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:09.762629 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:09.764381 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:09.766091 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.766102 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.766108 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.766366 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.766636 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:09.766638 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:09.766639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:09.766719 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:09.767388 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:09.767397 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:09.767594 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 18:38:09.767757 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 18:38:09.767979 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":141,"threads_started":1}
I20260807 18:38:09.768246 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:09.750271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36264 (local address 127.6.138.126:42131)
0807 18:38:09.750417 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:09.750421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:09.750623 (+ 202us) server_negotiation.cc:407] Connection header received
0807 18:38:09.750730 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:09.750732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:09.750775 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:09.750848 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:09.751529 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.752251 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.753131 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:09.753343 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:09.753431 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:09.764598 (+ 11167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:09.764602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:09.764605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:09.764636 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:09.765967 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.766464 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.766467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.766468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.766549 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:09.766871 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:09.766875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:09.766876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:09.767204 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:09.767318 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:09.767832 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 18:38:09.768074 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":58}
I20260807 18:38:09.772802 8576 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:36264:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:38:09.773752 8576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:09.779587 9041 tablet_service.cc:1467] Processing CreateTablet for tablet 81f391969a314278b190fb30b4a09ee0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cbfe10a175ca41f997fa6676190b0bda]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:38:09.779557 8808 tablet_service.cc:1467] Processing CreateTablet for tablet 81f391969a314278b190fb30b4a09ee0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cbfe10a175ca41f997fa6676190b0bda]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:38:09.779685 9260 tablet_service.cc:1467] Processing CreateTablet for tablet 81f391969a314278b190fb30b4a09ee0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cbfe10a175ca41f997fa6676190b0bda]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:38:09.780030 9041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f391969a314278b190fb30b4a09ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:09.780031 8808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f391969a314278b190fb30b4a09ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:09.780031 9260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f391969a314278b190fb30b4a09ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:09.783329 9402 tablet_bootstrap.cc:492] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: Bootstrap starting.
I20260807 18:38:09.783712 9403 tablet_bootstrap.cc:492] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: Bootstrap starting.
I20260807 18:38:09.784703 9403 tablet_bootstrap.cc:654] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:09.785288 9402 tablet_bootstrap.cc:654] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:09.786744 9404 tablet_bootstrap.cc:492] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: Bootstrap starting.
I20260807 18:38:09.787442 9403 tablet_bootstrap.cc:492] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: No bootstrap required, opened a new log
I20260807 18:38:09.787555 9403 ts_tablet_manager.cc:1397] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:38:09.787569 9402 tablet_bootstrap.cc:492] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: No bootstrap required, opened a new log
I20260807 18:38:09.787659 9402 ts_tablet_manager.cc:1397] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:38:09.787895 9404 tablet_bootstrap.cc:654] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:09.788035 9403 raft_consensus.cc:348] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.788064 9402 raft_consensus.cc:348] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.788163 9403 raft_consensus.cc:374] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:09.788195 9402 raft_consensus.cc:374] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:09.788198 9403 raft_consensus.cc:729] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b5eafe5e514bdba1f81683f6077740, State: Initialized, Role: FOLLOWER
I20260807 18:38:09.788231 9402 raft_consensus.cc:729] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d2dc2700f941be838bf5a35081c0e6, State: Initialized, Role: FOLLOWER
I20260807 18:38:09.788378 9402 consensus_queue.cc:260] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.788653 9402 ts_tablet_manager.cc:1428] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 18:38:09.788750 9412 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:09.788939 9403 consensus_queue.cc:260] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.789028 9412 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:09.789227 9403 ts_tablet_manager.cc:1428] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:09.789430 9413 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:09.789552 9413 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:09.790192 9404 tablet_bootstrap.cc:492] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: No bootstrap required, opened a new log
I20260807 18:38:09.790273 9404 ts_tablet_manager.cc:1397] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 18:38:09.790683 9404 raft_consensus.cc:348] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.791700 9404 raft_consensus.cc:374] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:09.791755 9404 raft_consensus.cc:729] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af8ae62212364cb1b363ecf9df7bbc63, State: Initialized, Role: FOLLOWER
I20260807 18:38:09.791913 9404 consensus_queue.cc:260] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:09.792213 9404 ts_tablet_manager.cc:1428] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 18:38:09.792234 9417 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:09.792421 9417 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:10.079484 9413 consensus_queue.cc:1035] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:10.083490 9462 consensus_queue.cc:1035] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:10.121315 9417 raft_consensus.cc:482] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:10.121501 9417 raft_consensus.cc:504] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:10.122673 9417 leader_election.cc:288] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1d2dc2700f941be838bf5a35081c0e6 (127.6.138.65:41611), a9b5eafe5e514bdba1f81683f6077740 (127.6.138.66:46571)
I20260807 18:38:10.127310 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.122965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:41611 (local address 127.6.138.67:59381)
0807 18:38:10.123426 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:10.123462 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:10.123591 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:10.123871 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:10.123874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:10.123895 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:10.124046 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.124053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.124931 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.124936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.126000 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.126010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.126830 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.126844 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.126957 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.127005 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 18:38:10.127085 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:38:10.127152 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":154,"thread_start_us":90,"threads_started":1}
I20260807 18:38:10.127341 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.123076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:38347 (local address 127.6.138.66:46571)
0807 18:38:10.123360 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:10.123365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:10.123731 (+ 366us) server_negotiation.cc:407] Connection header received
0807 18:38:10.123791 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:10.123793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:10.123866 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:10.123953 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:10.124308 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.124886 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.126081 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.126635 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.126770 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.126835 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:10.127021 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 18:38:10.127134 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":65,"threads_started":1}
I20260807 18:38:10.127341 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.123206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:59381 (local address 127.6.138.65:41611)
0807 18:38:10.123474 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:10.123479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:10.123558 (+ 79us) server_negotiation.cc:407] Connection header received
0807 18:38:10.123717 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:10.123720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:10.123766 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:10.123836 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:10.124174 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.124810 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.126148 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.126648 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.126788 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.126843 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:10.127124 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:38:10.127215 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260807 18:38:10.127326 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.122965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:46571 (local address 127.6.138.67:38347)
0807 18:38:10.123360 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:10.123379 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:10.123511 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:10.123990 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:10.123992 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:10.124008 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:10.124127 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.124137 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.124996 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.124999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.125957 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.125967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.126769 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.126772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.126897 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.126916 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:38:10.126979 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:38:10.127043 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":71,"thread_start_us":82,"threads_started":1}
I20260807 18:38:10.128304 9045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f391969a314278b190fb30b4a09ee0" candidate_uuid: "af8ae62212364cb1b363ecf9df7bbc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b5eafe5e514bdba1f81683f6077740" is_pre_election: true
I20260807 18:38:10.128458 8843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f391969a314278b190fb30b4a09ee0" candidate_uuid: "af8ae62212364cb1b363ecf9df7bbc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d2dc2700f941be838bf5a35081c0e6" is_pre_election: true
I20260807 18:38:10.128593 8843 raft_consensus.cc:2454] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af8ae62212364cb1b363ecf9df7bbc63 in term 0.
I20260807 18:38:10.128623 9045 raft_consensus.cc:2454] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af8ae62212364cb1b363ecf9df7bbc63 in term 0.
I20260807 18:38:10.129024 9188 leader_election.cc:302] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af8ae62212364cb1b363ecf9df7bbc63, e1d2dc2700f941be838bf5a35081c0e6; no voters:
I20260807 18:38:10.129284 9417 raft_consensus.cc:2781] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:10.129377 9417 raft_consensus.cc:482] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:10.129433 9417 raft_consensus.cc:3037] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:10.130545 9417 raft_consensus.cc:504] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:10.130950 9417 leader_election.cc:288] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [CANDIDATE]: Term 1 election: Requested vote from peers e1d2dc2700f941be838bf5a35081c0e6 (127.6.138.65:41611), a9b5eafe5e514bdba1f81683f6077740 (127.6.138.66:46571)
I20260807 18:38:10.131330 9045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f391969a314278b190fb30b4a09ee0" candidate_uuid: "af8ae62212364cb1b363ecf9df7bbc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9b5eafe5e514bdba1f81683f6077740"
I20260807 18:38:10.131353 8843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f391969a314278b190fb30b4a09ee0" candidate_uuid: "af8ae62212364cb1b363ecf9df7bbc63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d2dc2700f941be838bf5a35081c0e6"
I20260807 18:38:10.131450 9045 raft_consensus.cc:3037] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:10.131502 8843 raft_consensus.cc:3037] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:10.132608 9045 raft_consensus.cc:2454] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af8ae62212364cb1b363ecf9df7bbc63 in term 1.
I20260807 18:38:10.132628 8843 raft_consensus.cc:2454] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af8ae62212364cb1b363ecf9df7bbc63 in term 1.
I20260807 18:38:10.132949 9187 leader_election.cc:302] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b5eafe5e514bdba1f81683f6077740, af8ae62212364cb1b363ecf9df7bbc63; no voters:
I20260807 18:38:10.133162 9417 raft_consensus.cc:2781] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:10.133400 9417 raft_consensus.cc:686] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 LEADER]: Becoming Leader. State: Replica: af8ae62212364cb1b363ecf9df7bbc63, State: Running, Role: LEADER
I20260807 18:38:10.133661 9417 consensus_queue.cc:237] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } }
I20260807 18:38:10.135129 9477 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: TxnStatusTablet state changed. Reason: New leader af8ae62212364cb1b363ecf9df7bbc63. Latest consensus state: current_term: 1 leader_uuid: "af8ae62212364cb1b363ecf9df7bbc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:10.135264 9477 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:38:10.135677 9480 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:38:10.135908 9480 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:38:10.136874 8578 catalog_manager.cc:5495] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 reported cstate change: term changed from 0 to 1, leader changed from <none> to af8ae62212364cb1b363ecf9df7bbc63 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "af8ae62212364cb1b363ecf9df7bbc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } health_report { overall_health: HEALTHY } } }
I20260807 18:38:10.166265 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.161650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.6.138.67:42991)
0807 18:38:10.162205 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:10.162211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:10.162223 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:38:10.162301 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:10.162303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:10.162353 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:10.162444 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:10.162772 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.163343 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.164647 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.164896 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.164980 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.165360 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:10.165837 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:10.165926 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:10.166066 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 18:38:10.166122 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":122,"threads_started":1}
I20260807 18:38:10.166289 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.161496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:42991 (local address 127.0.0.1:60008)
0807 18:38:10.161865 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:10.161879 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:10.161979 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:10.162510 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:10.162513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:10.162523 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:10.162647 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.162654 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.163466 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.163469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.164514 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.164526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.165028 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.165032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.165129 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.165193 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:10.165942 (+ 749us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:10.165961 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:38:10.166040 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:38:10.166140 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237}
I20260807 18:38:10.169704 8843 raft_consensus.cc:1261] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6 [term 1 FOLLOWER]: Refusing update from remote peer af8ae62212364cb1b363ecf9df7bbc63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:10.169904 9045 raft_consensus.cc:1261] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740 [term 1 FOLLOWER]: Refusing update from remote peer af8ae62212364cb1b363ecf9df7bbc63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:10.170354 9477 consensus_queue.cc:1035] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:10.170544 9417 consensus_queue.cc:1035] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:10.172207 9412 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: TxnStatusTablet state changed. Reason: New leader af8ae62212364cb1b363ecf9df7bbc63. Latest consensus state: current_term: 1 leader_uuid: "af8ae62212364cb1b363ecf9df7bbc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:10.172372 9412 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:10.174289 9413 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: TxnStatusTablet state changed. Reason: New leader af8ae62212364cb1b363ecf9df7bbc63. Latest consensus state: current_term: 1 leader_uuid: "af8ae62212364cb1b363ecf9df7bbc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:10.174475 9413 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:10.176108 9417 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af8ae62212364cb1b363ecf9df7bbc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:10.176231 9417 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:38:10.177925 9412 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af8ae62212364cb1b363ecf9df7bbc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:10.178130 9412 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P e1d2dc2700f941be838bf5a35081c0e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:10.179131 9477 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af8ae62212364cb1b363ecf9df7bbc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:10.179265 9477 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P af8ae62212364cb1b363ecf9df7bbc63: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:38:10.179973 9461 mvcc.cc:205] Tried to move back new op lower bound from 7315979838129106944 to 7315979837989953536. Current Snapshot: MvccSnapshot[applied={T|T < 7315979838129106944}]
I20260807 18:38:10.180176 9413 tablet_replica.cc:439] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af8ae62212364cb1b363ecf9df7bbc63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1d2dc2700f941be838bf5a35081c0e6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41611 } } peers { permanent_uuid: "a9b5eafe5e514bdba1f81683f6077740" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46571 } } peers { permanent_uuid: "af8ae62212364cb1b363ecf9df7bbc63" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42991 } } }
I20260807 18:38:10.180745 9413 tablet_replica.cc:442] T 81f391969a314278b190fb30b4a09ee0 P a9b5eafe5e514bdba1f81683f6077740: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:10.194726 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.190860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:52237 (local address 127.6.138.67:42991)
0807 18:38:10.191018 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:10.191022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:10.191135 (+ 113us) server_negotiation.cc:407] Connection header received
0807 18:38:10.191280 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:10.191282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:10.191331 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:10.191397 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:10.191684 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.192200 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.193624 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.193861 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.193943 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.194276 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:10.194351 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:10.194407 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:10.194550 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 18:38:10.194608 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:38:10.194790 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.190691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:42991 (local address 127.6.138.66:52237)
0807 18:38:10.191043 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:10.191059 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:10.191174 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:10.191446 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:10.191449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:10.191458 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:10.191580 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.191586 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.192359 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.192362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.193471 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.193487 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.193976 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.193980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.194073 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.194128 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:10.194445 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:10.194452 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:38:10.194527 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 18:38:10.194623 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":141,"threads_started":1}
I20260807 18:38:10.225042 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.220444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:54813 (local address 127.6.138.66:46571)
0807 18:38:10.220570 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:10.220575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:10.220983 (+ 408us) server_negotiation.cc:407] Connection header received
0807 18:38:10.221153 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:10.221156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:10.221221 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:10.221304 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:10.221577 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.222408 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.223737 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.224072 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.224198 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.224562 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:10.224654 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:10.224742 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:10.224832 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 18:38:10.224891 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:38:10.225057 9509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.220394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:46571 (local address 127.6.138.67:54813)
0807 18:38:10.220882 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:10.220895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:10.221032 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:10.221322 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:10.221324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:10.221331 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:10.221472 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.221477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.222555 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.222559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.223572 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:10.223584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.224232 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.224238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:10.224363 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.224429 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:10.224746 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:10.224755 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:38:10.224821 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 18:38:10.224902 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":121,"threads_started":1}
I20260807 18:38:10.251566 8576 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:10.251722 8576 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:10.253999 8576 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82d9e9de011f453ebd9a073eca49e780: Sending DeleteTablet for 3 replicas of tablet 1fe4fc7cc27047208a22289dd9d71625
I20260807 18:38:10.254592 9041 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe4fc7cc27047208a22289dd9d71625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:10 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:42872
I20260807 18:38:10.254703 8808 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe4fc7cc27047208a22289dd9d71625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:10 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:43022
I20260807 18:38:10.254999 9260 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe4fc7cc27047208a22289dd9d71625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:10 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:60004
I20260807 18:38:10.255097 9513 tablet_replica.cc:330] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: stopping tablet replica
I20260807 18:38:10.255136 9514 tablet_replica.cc:330] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: stopping tablet replica
I20260807 18:38:10.255256 9513 raft_consensus.cc:2229] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:10.255264 9514 raft_consensus.cc:2229] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:10.255383 9514 raft_consensus.cc:2258] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:10.255457 9513 raft_consensus.cc:2258] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:10.256138 9514 ts_tablet_manager.cc:1905] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:10.256156 9513 ts_tablet_manager.cc:1905] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:10.256580 9516 tablet_replica.cc:330] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: stopping tablet replica
I20260807 18:38:10.257418 9516 raft_consensus.cc:2229] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:10.257620 9516 raft_consensus.cc:2258] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:10.258370 9516 ts_tablet_manager.cc:1905] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:10.259398 9513 ts_tablet_manager.cc:1918] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:10.259397 9514 ts_tablet_manager.cc:1918] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:10.259480 9514 log.cc:1196] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/1fe4fc7cc27047208a22289dd9d71625
I20260807 18:38:10.259483 9513 log.cc:1196] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/1fe4fc7cc27047208a22289dd9d71625
I20260807 18:38:10.259804 9513 ts_tablet_manager.cc:1939] T 1fe4fc7cc27047208a22289dd9d71625 P a9b5eafe5e514bdba1f81683f6077740: Deleting consensus metadata
I20260807 18:38:10.259845 9514 ts_tablet_manager.cc:1939] T 1fe4fc7cc27047208a22289dd9d71625 P e1d2dc2700f941be838bf5a35081c0e6: Deleting consensus metadata
I20260807 18:38:10.260600 8556 catalog_manager.cc:4841] TS a9b5eafe5e514bdba1f81683f6077740 (127.6.138.66:46571): tablet 1fe4fc7cc27047208a22289dd9d71625 (table test-table [id=6c11e942da8c469e8506e51755325878]) successfully deleted
I20260807 18:38:10.260684 8557 catalog_manager.cc:4841] TS e1d2dc2700f941be838bf5a35081c0e6 (127.6.138.65:41611): tablet 1fe4fc7cc27047208a22289dd9d71625 (table test-table [id=6c11e942da8c469e8506e51755325878]) successfully deleted
I20260807 18:38:10.261795 9516 ts_tablet_manager.cc:1918] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:10.261901 9516 log.cc:1196] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/1fe4fc7cc27047208a22289dd9d71625
I20260807 18:38:10.262257 9516 ts_tablet_manager.cc:1939] T 1fe4fc7cc27047208a22289dd9d71625 P af8ae62212364cb1b363ecf9df7bbc63: Deleting consensus metadata
I20260807 18:38:10.262976 8555 catalog_manager.cc:4841] TS af8ae62212364cb1b363ecf9df7bbc63 (127.6.138.67:42991): tablet 1fe4fc7cc27047208a22289dd9d71625 (table test-table [id=6c11e942da8c469e8506e51755325878]) successfully deleted
Aug 07 18:38:10 dist-test-slave-2d19 krb5kdc[8504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127889, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:10.272377 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.257011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.6.138.65:41611)
0807 18:38:10.257127 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:10.257135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:10.262261 (+ 5126us) server_negotiation.cc:407] Connection header received
0807 18:38:10.262457 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:10.262461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:10.262520 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:10.262616 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:10.263511 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.264012 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.265230 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.265440 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.265522 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.268582 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:10.268602 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:10.268619 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:10.268645 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:10.270460 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:10.270954 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:10.270959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:10.270961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:10.271045 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:10.271350 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:10.271354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:10.271355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:10.271622 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:10.271873 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:10.272042 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 18:38:10.272184 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 18:38:10.272848 8901 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43026
I20260807 18:38:10.283037 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:10.274658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.6.138.126:42131)
0807 18:38:10.274908 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:10.274911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:10.274924 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:38:10.274963 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:10.274965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:10.275005 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:10.275081 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:10.275863 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.276560 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.277671 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:10.277891 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:10.277974 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:10.279602 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:10.279611 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:10.279614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:10.279638 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:10.281284 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:10.281792 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:10.281794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:10.281796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:10.281874 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:10.282180 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:10.282182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:10.282183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:10.282349 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:10.282455 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:10.282733 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 18:38:10.282845 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":102,"threads_started":1}
W20260807 18:38:10.283464 8576 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36274
I20260807 18:38:10.285079 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 8702
I20260807 18:38:10.309371 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 8935
I20260807 18:38:10.326181 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 9163
I20260807 18:38:10.340636 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 8524
2026-08-07T18:38:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3438 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:10 dist-test-slave-2d19 krb5kdc[9528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:10 dist-test-slave-2d19 krb5kdc[9528](info): set up 2 sockets
Aug 07 18:38:10 dist-test-slave-2d19 krb5kdc[9528](info): commencing operation
krb5kdc: starting...
W20260807 18:38:11.622407 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:11 dist-test-slave-2d19 krb5kdc[9528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:11.755213 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42309
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:11.873808 9546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:11.874150 9546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:11.874239 9546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:11.878376 9546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:11.878469 9546 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:38:11.878501 9546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:11.878548 9546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:11.878585 9546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:11.878620 9546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:11.884075 9546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42309
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:11.885269 9546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:11.886343 9546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:11.895183 9551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:11.895210 9552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:11.895296 9546 server_base.cc:1034] running on GCE node
W20260807 18:38:11.895236 9555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:11.896101 9546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:11.897292 9546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:11.898506 9546 hybrid_clock.cc:648] HybridClock initialized: now 1786127891898474 us; error 46 us; skew 500 ppm
Aug 07 18:38:11 dist-test-slave-2d19 krb5kdc[9528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127891, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:11.906932 9546 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:11.907897 9546 webserver.cc:460] Webserver started at http://127.6.138.126:44983/ using document root <none> and password file <none>
I20260807 18:38:11.908586 9546 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:11.908687 9546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:11.908937 9546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:11.911023 9546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "48b10794060949e5898c429e32d4c2d4"
format_stamp: "Formatted at 2026-08-07 18:38:11 on dist-test-slave-2d19"
server_key: "72fcb9c726e6e7ae8f773f723c29452e"
server_key_iv: "08edb5eb1a7f594c65918173f796536d"
server_key_version: "encryptionkey@0"
I20260807 18:38:11.911651 9546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "48b10794060949e5898c429e32d4c2d4"
format_stamp: "Formatted at 2026-08-07 18:38:11 on dist-test-slave-2d19"
server_key: "72fcb9c726e6e7ae8f773f723c29452e"
server_key_iv: "08edb5eb1a7f594c65918173f796536d"
server_key_version: "encryptionkey@0"
I20260807 18:38:11.915936 9546 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 18:38:11.918735 9563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:11.920027 9546 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:38:11.920182 9546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "48b10794060949e5898c429e32d4c2d4"
format_stamp: "Formatted at 2026-08-07 18:38:11 on dist-test-slave-2d19"
server_key: "72fcb9c726e6e7ae8f773f723c29452e"
server_key_iv: "08edb5eb1a7f594c65918173f796536d"
server_key_version: "encryptionkey@0"
I20260807 18:38:11.920384 9546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:11.932687 9546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:11.935798 9546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:11.936031 9546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:11.944660 9546 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:42309
I20260807 18:38:11.944686 9663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:42309 every 8 connection(s)
I20260807 18:38:11.945634 9546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:11.948578 9664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:11.952491 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 9546
I20260807 18:38:11.952628 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:11.952881 6697 external_mini_cluster.cc:1428] Setting key 58d693ed0ccccd84a55d155816036f04
I20260807 18:38:11.954583 9664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: Bootstrap starting.
I20260807 18:38:11.957150 9664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:11.957908 9664 log.cc:824] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:11.960227 9664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: No bootstrap required, opened a new log
Aug 07 18:38:11 dist-test-slave-2d19 krb5kdc[9528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:11.963984 9664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b10794060949e5898c429e32d4c2d4" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42309 } }
I20260807 18:38:11.964263 9664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:11.964367 9664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b10794060949e5898c429e32d4c2d4, State: Initialized, Role: FOLLOWER
I20260807 18:38:11.964866 9664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b10794060949e5898c429e32d4c2d4" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42309 } }
I20260807 18:38:11.965029 9664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:11.965132 9664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:11.965261 9664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:11.966163 9664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b10794060949e5898c429e32d4c2d4" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42309 } }
I20260807 18:38:11.966538 9664 leader_election.cc:302] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48b10794060949e5898c429e32d4c2d4; no voters:
I20260807 18:38:11.966924 9664 leader_election.cc:288] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:11.967051 9674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:11.967300 9674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [term 1 LEADER]: Becoming Leader. State: Replica: 48b10794060949e5898c429e32d4c2d4, State: Running, Role: LEADER
I20260807 18:38:11.967660 9674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b10794060949e5898c429e32d4c2d4" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42309 } }
I20260807 18:38:11.967950 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:11.954232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53222 (local address 127.6.138.126:42309)
0807 18:38:11.954649 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:11.954665 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:11.954722 (+ 57us) server_negotiation.cc:407] Connection header received
0807 18:38:11.955362 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:11.955378 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:11.955705 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:11.955993 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:11.956921 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:11.958008 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:11.959082 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:11.959556 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:11.959692 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:11.962665 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:11.962671 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:11.962686 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:11.962713 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:11.964623 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:11.965067 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:11.965073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:11.965082 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:11.965247 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:11.965574 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:11.965577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:11.965579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:11.965996 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:11.966209 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:11.966503 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 18:38:11.966839 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":139,"threads_started":1}
I20260807 18:38:11.968273 9664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:11.969555 9677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48b10794060949e5898c429e32d4c2d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b10794060949e5898c429e32d4c2d4" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42309 } } }
I20260807 18:38:11.969539 9679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48b10794060949e5898c429e32d4c2d4. Latest consensus state: current_term: 1 leader_uuid: "48b10794060949e5898c429e32d4c2d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b10794060949e5898c429e32d4c2d4" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42309 } } }
I20260807 18:38:11.969688 9677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:11.969688 9679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4 [sys.catalog]: This master's current role is: LEADER
W20260807 18:38:11.973109 9702 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:38:11.973204 9702 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:38:11.973290 9689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:11.974068 9689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:11.979915 9689 catalog_manager.cc:1347] Generated new cluster ID: 8d52ae499a0e4ee98ed9fa1e21060041
I20260807 18:38:11.979986 9689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:11.990239 9689 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:11.991185 9689 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:12.003068 9689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: Generated new TSK 0
I20260807 18:38:12.003803 9689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:12.043653 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42309
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:12.179327 9711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:12.179600 9711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:12.179651 9711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:12.183918 9711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:12.183982 9711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:12.184023 9711 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:38:12.184037 9711 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:38:12.184070 9711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:12.189460 9711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.138.126:42309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:12.190711 9711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:12.191780 9711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:12.201695 9725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:12.201766 9722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:12.201788 9723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:12.201893 9711 server_base.cc:1034] running on GCE node
I20260807 18:38:12.202510 9711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:12.203197 9711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:12.204344 9711 hybrid_clock.cc:648] HybridClock initialized: now 1786127892204329 us; error 25 us; skew 500 ppm
Aug 07 18:38:12 dist-test-slave-2d19 krb5kdc[9528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127892, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:12.207217 9711 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:12.208030 9711 webserver.cc:460] Webserver started at http://127.6.138.65:39297/ using document root <none> and password file <none>
I20260807 18:38:12.208632 9711 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:12.208696 9711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:12.208863 9711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:12.210841 9711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "e8a64bab799747d8a397357fd09181c8"
format_stamp: "Formatted at 2026-08-07 18:38:12 on dist-test-slave-2d19"
server_key: "b583d950ad2e37059615471060da432c"
server_key_iv: "549cd49bf0503e36749136d8def74442"
server_key_version: "encryptionkey@0"
I20260807 18:38:12.211417 9711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "e8a64bab799747d8a397357fd09181c8"
format_stamp: "Formatted at 2026-08-07 18:38:12 on dist-test-slave-2d19"
server_key: "b583d950ad2e37059615471060da432c"
server_key_iv: "549cd49bf0503e36749136d8def74442"
server_key_version: "encryptionkey@0"
I20260807 18:38:12.215153 9711 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 18:38:12.217671 9736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:12.218894 9711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:12.219051 9711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "e8a64bab799747d8a397357fd09181c8"
format_stamp: "Formatted at 2026-08-07 18:38:12 on dist-test-slave-2d19"
server_key: "b583d950ad2e37059615471060da432c"
server_key_iv: "549cd49bf0503e36749136d8def74442"
server_key_version: "encryptionkey@0"
I20260807 18:38:12.219168 9711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:12.245455 9711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:12.251833 9711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:12.252082 9711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:12.254220 9711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:12.254325 9711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:12.254400 9711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:12.254473 9711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:38:12 dist-test-slave-2d19 krb5kdc[9528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127892, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:12.268906 9711 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:41871
I20260807 18:38:12.269004 9926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:41871 every 8 connection(s)
I20260807 18:38:12.270337 9711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:12.272293 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 9711
I20260807 18:38:12.272454 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:12.272799 6697 external_mini_cluster.cc:1428] Setting key 9fa9f37a87041d2fbc3f6d3a4af06906
I20260807 18:38:12.273694 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.255039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:36543 (local address 127.6.138.126:42309)
0807 18:38:12.255175 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.255184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.257024 (+ 1840us) server_negotiation.cc:407] Connection header received
0807 18:38:12.258534 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.258537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.258590 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.258653 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:12.260560 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.261425 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.263306 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.263689 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.263835 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.269156 (+ 5321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:12.269160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:12.269163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:12.269202 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:12.271152 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.271743 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.271746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.271747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.271827 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.272233 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.272235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.272236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.272568 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:12.272657 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.273346 (+ 689us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.273514 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:38:12.274573 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.256223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42309 (local address 127.6.138.65:36543)
0807 18:38:12.256839 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:12.256902 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:12.258214 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:12.259097 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:12.259109 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:12.259616 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:12.260332 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.260355 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.261633 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.261639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.263133 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.263145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.263899 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.263904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.264136 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.266635 (+ 2499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:12.266657 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:12.268904 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:12.271290 (+ 2386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.271297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.271311 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.271631 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.271904 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.271905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.271907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.272126 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.272698 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:12.272705 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:12.273030 (+ 325us) client_negotiation.cc:769] Sending connection context
0807 18:38:12.273259 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 18:38:12.273535 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":145,"threads_started":1}
I20260807 18:38:12.276083 9927 heartbeater.cc:347] Connected to a master server at 127.6.138.126:42309
I20260807 18:38:12.276302 9927 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:12.276947 9927 heartbeater.cc:511] Master 127.6.138.126:42309 requested a full tablet report, sending...
I20260807 18:38:12.278877 9591 ts_manager.cc:194] Registered new tserver with Master: e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871)
I20260807 18:38:12.280349 9591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:36543
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:38:12.287070 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.278446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42309 (local address 127.6.138.65:38445)
0807 18:38:12.278803 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:12.278819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:12.278927 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:12.279298 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:12.279301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:12.279524 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:12.279657 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.279666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.280892 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.280896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.281640 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.281651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.282139 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.282142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.282344 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.283529 (+ 1185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:12.283541 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:12.283882 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:12.285728 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.285731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.285732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.285935 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.286222 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.286224 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.286225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.286301 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.286641 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:12.286643 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:12.286721 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 18:38:12.286803 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 18:38:12.286911 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":23,"thread_start_us":88,"threads_started":1}
I20260807 18:38:12.287076 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.278527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:38445 (local address 127.6.138.126:42309)
0807 18:38:12.278640 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.278644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.278909 (+ 265us) server_negotiation.cc:407] Connection header received
0807 18:38:12.279099 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.279105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.279163 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.279251 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:12.279799 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.280737 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.281772 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.282000 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.282112 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.284028 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:12.284034 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:12.284042 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:12.284064 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:12.285588 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.286035 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.286038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.286039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.286122 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.286381 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.286383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.286384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.286553 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:12.286649 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.286863 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.286964 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:12.305301 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42309
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:12.422308 9947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:12.422631 9947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:12.422711 9947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:12.426931 9947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:12.427050 9947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:12.427130 9947 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:38:12.427178 9947 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:38:12.427244 9947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:12.432827 9947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.138.126:42309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:12.434003 9947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:12.435081 9947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:12.445065 9955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:12.445134 9947 server_base.cc:1034] running on GCE node
W20260807 18:38:12.445176 9953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:12.445082 9962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:12.446107 9947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:12.446795 9947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:12.447978 9947 hybrid_clock.cc:648] HybridClock initialized: now 1786127892447953 us; error 27 us; skew 500 ppm
Aug 07 18:38:12 dist-test-slave-2d19 krb5kdc[9528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127892, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:12.451004 9947 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:12.451926 9947 webserver.cc:460] Webserver started at http://127.6.138.66:42247/ using document root <none> and password file <none>
I20260807 18:38:12.452600 9947 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:12.452695 9947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:12.452943 9947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:12.455044 9947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "4773ee8d20364403ad609186320e117d"
format_stamp: "Formatted at 2026-08-07 18:38:12 on dist-test-slave-2d19"
server_key: "cd0fa096e4a31ccf9af533e2945c9f25"
server_key_iv: "e09329575cc0dfe5d0c46072f488340d"
server_key_version: "encryptionkey@0"
I20260807 18:38:12.455634 9947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "4773ee8d20364403ad609186320e117d"
format_stamp: "Formatted at 2026-08-07 18:38:12 on dist-test-slave-2d19"
server_key: "cd0fa096e4a31ccf9af533e2945c9f25"
server_key_iv: "e09329575cc0dfe5d0c46072f488340d"
server_key_version: "encryptionkey@0"
I20260807 18:38:12.459506 9947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 18:38:12.462067 9971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:12.463286 9947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:12.463426 9947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "4773ee8d20364403ad609186320e117d"
format_stamp: "Formatted at 2026-08-07 18:38:12 on dist-test-slave-2d19"
server_key: "cd0fa096e4a31ccf9af533e2945c9f25"
server_key_iv: "e09329575cc0dfe5d0c46072f488340d"
server_key_version: "encryptionkey@0"
I20260807 18:38:12.463547 9947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:12.489837 9947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:12.493100 9947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:12.493314 9947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:12.494869 9947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:12.494948 9947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:12.495048 9947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:12.495103 9947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:38:12 dist-test-slave-2d19 krb5kdc[9528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127892, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:12.506479 9947 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:41793
I20260807 18:38:12.506670 10155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:41793 every 8 connection(s)
I20260807 18:38:12.507445 9947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:12.509969 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.496012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:54895 (local address 127.6.138.126:42309)
0807 18:38:12.496184 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.496190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.497203 (+ 1013us) server_negotiation.cc:407] Connection header received
0807 18:38:12.498612 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.498618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.498687 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.498792 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:12.500519 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.501140 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.502322 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.502846 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.502979 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.505947 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:12.505950 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:12.505953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:12.505993 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:12.507679 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.508332 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.508335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.508337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.508434 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.508811 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.508814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.508816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.509020 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:12.509135 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.509654 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.509783 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:38:12.510746 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.496487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42309 (local address 127.6.138.66:54895)
0807 18:38:12.497013 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:12.497076 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:12.498325 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:12.499020 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:12.499031 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:12.499560 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:12.500340 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.500365 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.501275 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.501279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.502188 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.502197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.503021 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.503026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.503305 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.504031 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:12.504047 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:12.505770 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:12.507798 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.507805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.507819 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.508192 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.508536 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.508540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.508542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.508702 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.509136 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:12.509143 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:12.509407 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 18:38:12.509591 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 18:38:12.509918 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":130,"threads_started":1}
I20260807 18:38:12.512033 10157 heartbeater.cc:347] Connected to a master server at 127.6.138.126:42309
I20260807 18:38:12.512216 10157 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:12.512787 10157 heartbeater.cc:511] Master 127.6.138.126:42309 requested a full tablet report, sending...
I20260807 18:38:12.513024 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 9947
I20260807 18:38:12.513139 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:12.513460 6697 external_mini_cluster.cc:1428] Setting key e7258abcce8936e5b0df19c8be76b50f
I20260807 18:38:12.514070 9589 ts_manager.cc:194] Registered new tserver with Master: 4773ee8d20364403ad609186320e117d (127.6.138.66:41793)
I20260807 18:38:12.514861 9589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:54895
I20260807 18:38:12.523627 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.514611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:37921 (local address 127.6.138.126:42309)
0807 18:38:12.514731 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.514735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.515178 (+ 443us) server_negotiation.cc:407] Connection header received
0807 18:38:12.515297 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.515299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.515348 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.515446 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:12.516801 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.517502 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.518341 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.518698 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.518843 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.520039 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:12.520042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:12.520045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:12.520069 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:12.521683 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.522201 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.522204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.522206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.522309 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.522605 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.522609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.522611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.522836 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:12.523021 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.523327 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.523463 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:38:12.523635 10169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.514521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42309 (local address 127.6.138.66:37921)
0807 18:38:12.515097 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:12.515113 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:12.515215 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:12.516387 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:12.516391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:12.516578 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:12.516676 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.516682 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.517651 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.517654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.518211 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.518220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.518850 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.518854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.518974 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.519530 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:12.519541 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:12.519872 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:12.521804 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.521808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.521810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.522097 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.522411 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.522416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.522418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.522510 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.523021 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:12.523024 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:12.523135 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 18:38:12.523263 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 18:38:12.523424 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:12.544052 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42309
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:12.660609 10172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:12.660864 10172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:12.660912 10172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:12.664853 10172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:12.664909 10172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:12.664949 10172 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:38:12.664963 10172 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:38:12.664995 10172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:12.670661 10172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.138.126:42309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:12.671705 10172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:12.672731 10172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:12.682679 10181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:12.682621 10182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:12.682619 10185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:12.683009 10172 server_base.cc:1034] running on GCE node
I20260807 18:38:12.683579 10172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:12.684238 10172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:12.685483 10172 hybrid_clock.cc:648] HybridClock initialized: now 1786127892685459 us; error 27 us; skew 500 ppm
Aug 07 18:38:12 dist-test-slave-2d19 krb5kdc[9528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127892, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:12.688316 10172 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:12.689277 10172 webserver.cc:460] Webserver started at http://127.6.138.67:34209/ using document root <none> and password file <none>
I20260807 18:38:12.689872 10172 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:12.689965 10172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:12.690207 10172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:12.692400 10172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "74666dfde8e940509f5ebf53f1754017"
format_stamp: "Formatted at 2026-08-07 18:38:12 on dist-test-slave-2d19"
server_key: "06bebd8aae164c968e5fee6fc8c9d0fa"
server_key_iv: "7470987777547ec1e969979b2e6459c5"
server_key_version: "encryptionkey@0"
I20260807 18:38:12.693024 10172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "74666dfde8e940509f5ebf53f1754017"
format_stamp: "Formatted at 2026-08-07 18:38:12 on dist-test-slave-2d19"
server_key: "06bebd8aae164c968e5fee6fc8c9d0fa"
server_key_iv: "7470987777547ec1e969979b2e6459c5"
server_key_version: "encryptionkey@0"
I20260807 18:38:12.696760 10172 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:12.699201 10194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:12.700460 10172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:12.700608 10172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "74666dfde8e940509f5ebf53f1754017"
format_stamp: "Formatted at 2026-08-07 18:38:12 on dist-test-slave-2d19"
server_key: "06bebd8aae164c968e5fee6fc8c9d0fa"
server_key_iv: "7470987777547ec1e969979b2e6459c5"
server_key_version: "encryptionkey@0"
I20260807 18:38:12.700719 10172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:12.710229 10172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:12.713672 10172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:12.713882 10172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:12.715380 10172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:12.715452 10172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:12.715547 10172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:12.715596 10172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:38:12 dist-test-slave-2d19 krb5kdc[9528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127892, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:12.727133 10172 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:37691
I20260807 18:38:12.727178 10371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:37691 every 8 connection(s)
I20260807 18:38:12.728364 10172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:12.730454 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.716116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:56839 (local address 127.6.138.126:42309)
0807 18:38:12.716309 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.716315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.717758 (+ 1443us) server_negotiation.cc:407] Connection header received
0807 18:38:12.718897 (+ 1139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.718901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.718957 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.719051 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:12.720746 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.721432 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.722902 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.723343 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.723477 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.726571 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:12.726577 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:12.726583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:12.726621 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:12.728447 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.729027 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.729030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.729032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.729131 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.729505 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.729508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.729510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.729696 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:12.729776 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.730235 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.730336 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:38:12.731307 10209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.717034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42309 (local address 127.6.138.67:56839)
0807 18:38:12.717598 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:12.717659 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:12.718680 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:12.719261 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:12.719270 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:12.719790 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:12.720582 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.720604 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.721577 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.721583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.722726 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.722758 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.723521 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.723528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.723749 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.724490 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:12.724511 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:12.726370 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:12.728569 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.728581 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.728592 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.728899 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.729242 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.729246 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.729247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.729383 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.729791 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:12.729796 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:12.730016 (+ 220us) client_negotiation.cc:769] Sending connection context
0807 18:38:12.730170 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:38:12.730414 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":173,"threads_started":1}
I20260807 18:38:12.731397 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 10172
I20260807 18:38:12.731532 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:12.731856 6697 external_mini_cluster.cc:1428] Setting key 2c9497a0843c66bca475c445e2e3fad0
I20260807 18:38:12.732837 10374 heartbeater.cc:347] Connected to a master server at 127.6.138.126:42309
I20260807 18:38:12.732992 10374 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:12.733579 10374 heartbeater.cc:511] Master 127.6.138.126:42309 requested a full tablet report, sending...
I20260807 18:38:12.734650 9591 ts_manager.cc:194] Registered new tserver with Master: 74666dfde8e940509f5ebf53f1754017 (127.6.138.67:37691)
I20260807 18:38:12.735432 9591 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:56839
I20260807 18:38:12.741899 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.735044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:60007 (local address 127.6.138.126:42309)
0807 18:38:12.735149 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.735154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.735395 (+ 241us) server_negotiation.cc:407] Connection header received
0807 18:38:12.735497 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.735499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.735536 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.735600 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:12.736069 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.736653 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.737682 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.737879 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.737952 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.738918 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:12.738921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:12.738923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:12.738945 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:12.740436 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.740871 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.740873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.740875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.740947 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.741209 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.741211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.741212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.741365 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:12.741440 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.741706 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.741794 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 18:38:12.741907 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.734916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42309 (local address 127.6.138.67:60007)
0807 18:38:12.735305 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:12.735318 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:12.735416 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:12.735625 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:12.735628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:12.735825 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:12.735922 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.735928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.736789 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.736793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.737575 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.737584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.737983 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.737985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.738049 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.738493 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:12.738504 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:12.738789 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:12.740545 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.740548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.740550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.740769 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.741034 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:12.741037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:12.741038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:12.741124 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:12.741473 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:12.741475 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:12.741556 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 18:38:12.741648 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:38:12.741764 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":93,"threads_started":1}
I20260807 18:38:12.745801 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:38:12 dist-test-slave-2d19 krb5kdc[9528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:38:12 dist-test-slave-2d19 krb5kdc[9528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:12.774415 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.764921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53224 (local address 127.6.138.126:42309)
0807 18:38:12.765064 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.765068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.765081 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:38:12.765129 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.765131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.765169 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.765234 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:12.766058 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.766589 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.767632 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.767899 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.767979 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.770924 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:12.770928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:12.770931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:12.770966 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:12.772545 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.773016 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.773019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.773020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.773103 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:12.773420 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:12.773424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:12.773425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:12.773652 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:12.773783 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.774066 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.774218 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:38:12.777949 9591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:12.781337 9591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:12.797612 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.790846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:41871 (local address 127.0.0.1:42298)
0807 18:38:12.791360 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:12.791409 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:12.791544 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:12.792485 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:12.792492 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:12.792519 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:12.792696 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.792704 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.794348 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.794354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.795877 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.795889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.797039 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.797043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.797153 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.797199 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:38:12.797293 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 18:38:12.797428 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":57,"thread_start_us":114,"threads_started":1}
I20260807 18:38:12.798158 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.791178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53568 (local address 127.6.138.67:37691)
0807 18:38:12.791928 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.791941 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.792515 (+ 574us) server_negotiation.cc:407] Connection header received
0807 18:38:12.792782 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.792791 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.792984 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.793150 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:12.793523 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.794323 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.796488 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.797271 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.797347 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.797509 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.797908 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.798007 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":106,"threads_started":1}
I20260807 18:38:12.798491 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.791624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.6.138.65:41871)
0807 18:38:12.791928 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.791938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.791964 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:38:12.792049 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.792055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.792307 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.792863 (+ 556us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:12.792998 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.794200 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.796035 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.796876 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.797918 (+ 1042us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.798061 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.798111 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.798357 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260807 18:38:12.798518 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.792222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.6.138.66:41793)
0807 18:38:12.792725 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:12.792745 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:12.792761 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:12.792823 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:12.792827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:12.792976 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:12.793129 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:12.793376 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.794613 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.796172 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.797032 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.797172 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.797364 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:12.798250 (+ 886us) server_negotiation.cc:299] Negotiation successful
0807 18:38:12.798355 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":130,"threads_started":1}
I20260807 18:38:12.798554 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.790983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:37691 (local address 127.0.0.1:53568)
0807 18:38:12.792000 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:12.792016 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:12.792608 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:12.793278 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:12.793284 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:12.793305 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:12.793406 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.793410 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.794455 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.794468 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.796343 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.796353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.797610 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.797613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.797730 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.797763 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:38:12.798317 (+ 554us) client_negotiation.cc:240] Negotiation successful
0807 18:38:12.798396 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":636,"thread_start_us":118,"threads_started":1}
I20260807 18:38:12.798852 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:12.791683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:41793 (local address 127.0.0.1:45754)
0807 18:38:12.792294 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:12.792308 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:12.792474 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:12.793108 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:12.793111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:12.793128 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:12.793240 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.793250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.794754 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.794759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.795998 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:12.796011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:12.797444 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:12.797448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:12.797552 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:12.797575 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:38:12.797662 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 18:38:12.798699 (+ 1037us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260807 18:38:12.801602 10043 tablet_service.cc:1467] Processing CreateTablet for tablet b151aa39f56f4a6f9751ea08a8bfa511 (DEFAULT_TABLE table=test-table [id=116a45a52fd4410db3be27770a5ad68a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:12.801707 10279 tablet_service.cc:1467] Processing CreateTablet for tablet b151aa39f56f4a6f9751ea08a8bfa511 (DEFAULT_TABLE table=test-table [id=116a45a52fd4410db3be27770a5ad68a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:12.801693 9819 tablet_service.cc:1467] Processing CreateTablet for tablet b151aa39f56f4a6f9751ea08a8bfa511 (DEFAULT_TABLE table=test-table [id=116a45a52fd4410db3be27770a5ad68a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:12.802745 10043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b151aa39f56f4a6f9751ea08a8bfa511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:12.802747 9819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b151aa39f56f4a6f9751ea08a8bfa511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:12.802763 10279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b151aa39f56f4a6f9751ea08a8bfa511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:12.809795 10410 tablet_bootstrap.cc:492] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: Bootstrap starting.
I20260807 18:38:12.810973 10413 tablet_bootstrap.cc:492] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: Bootstrap starting.
I20260807 18:38:12.811326 10417 tablet_bootstrap.cc:492] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: Bootstrap starting.
I20260807 18:38:12.812302 10410 tablet_bootstrap.cc:654] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:12.813035 10413 tablet_bootstrap.cc:654] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:12.813220 10410 log.cc:824] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:12.813406 10417 tablet_bootstrap.cc:654] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:12.814047 10413 log.cc:824] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:12.814194 10417 log.cc:824] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:12.815317 10410 tablet_bootstrap.cc:492] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: No bootstrap required, opened a new log
I20260807 18:38:12.815526 10410 ts_tablet_manager.cc:1397] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:38:12.815852 10413 tablet_bootstrap.cc:492] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: No bootstrap required, opened a new log
I20260807 18:38:12.816043 10413 ts_tablet_manager.cc:1397] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:38:12.816960 10417 tablet_bootstrap.cc:492] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: No bootstrap required, opened a new log
I20260807 18:38:12.817155 10417 ts_tablet_manager.cc:1397] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:38:12.818768 10410 raft_consensus.cc:348] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } }
I20260807 18:38:12.819012 10410 raft_consensus.cc:374] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:12.819087 10410 raft_consensus.cc:729] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4773ee8d20364403ad609186320e117d, State: Initialized, Role: FOLLOWER
I20260807 18:38:12.819602 10410 consensus_queue.cc:260] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } }
I20260807 18:38:12.820736 10410 ts_tablet_manager.cc:1428] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 18:38:12.820708 10413 raft_consensus.cc:348] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } }
I20260807 18:38:12.820966 10413 raft_consensus.cc:374] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:12.821026 10413 raft_consensus.cc:729] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a64bab799747d8a397357fd09181c8, State: Initialized, Role: FOLLOWER
I20260807 18:38:12.821133 10157 heartbeater.cc:503] Master 127.6.138.126:42309 was elected leader, sending a full tablet report...
I20260807 18:38:12.821164 10417 raft_consensus.cc:348] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } }
I20260807 18:38:12.821404 10417 raft_consensus.cc:374] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:12.821472 10417 raft_consensus.cc:729] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74666dfde8e940509f5ebf53f1754017, State: Initialized, Role: FOLLOWER
I20260807 18:38:12.821513 10413 consensus_queue.cc:260] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } }
I20260807 18:38:12.821879 10417 consensus_queue.cc:260] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } }
I20260807 18:38:12.822194 9927 heartbeater.cc:503] Master 127.6.138.126:42309 was elected leader, sending a full tablet report...
I20260807 18:38:12.822511 10413 ts_tablet_manager.cc:1428] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 18:38:12.822952 10417 ts_tablet_manager.cc:1428] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:38:12.823975 10374 heartbeater.cc:503] Master 127.6.138.126:42309 was elected leader, sending a full tablet report...
W20260807 18:38:12.979463 10375 tablet.cc:2369] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:38:13.008759 10158 tablet.cc:2369] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:13.011257 10426 raft_consensus.cc:482] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:13.011513 10426 raft_consensus.cc:504] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } }
I20260807 18:38:13.012998 10426 leader_election.cc:288] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 4773ee8d20364403ad609186320e117d (127.6.138.66:41793)
I20260807 18:38:13.017643 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.013396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:53203 (local address 127.6.138.65:41871)
0807 18:38:13.013545 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.013552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.013616 (+ 64us) server_negotiation.cc:407] Connection header received
0807 18:38:13.013680 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.013684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.013741 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.013829 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:13.014155 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.015016 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.016413 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.017065 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.017193 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.017272 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.017464 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.017535 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:38:13.017652 10209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.013299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:41871 (local address 127.6.138.67:53203)
0807 18:38:13.013413 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:13.013429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:13.013538 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:13.013847 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:13.013849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:13.013863 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:13.014028 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.014037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.015157 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.015161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.016229 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.016239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.017192 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.017195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.017322 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.017349 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:38:13.017437 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 18:38:13.017513 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 18:38:13.018318 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b151aa39f56f4a6f9751ea08a8bfa511" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8" is_pre_election: true
I20260807 18:38:13.018460 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.013376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:49713 (local address 127.6.138.66:41793)
0807 18:38:13.013497 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.013501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.013967 (+ 466us) server_negotiation.cc:407] Connection header received
0807 18:38:13.014080 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.014084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.014144 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.014224 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:13.015030 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.015801 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.017295 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.017899 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.018027 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.018076 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.018281 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.018354 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:38:13.018469 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.013299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:41793 (local address 127.6.138.67:49713)
0807 18:38:13.013846 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:13.013860 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:13.013980 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:13.014253 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:13.014256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:13.014274 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:13.014826 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.014836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.015971 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.015973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.017169 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.017179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.018035 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.018038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.018154 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.018170 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 18:38:13.018262 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:38:13.018325 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":118,"thread_start_us":127,"threads_started":1}
I20260807 18:38:13.018754 9826 raft_consensus.cc:2454] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 0.
I20260807 18:38:13.019114 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b151aa39f56f4a6f9751ea08a8bfa511" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4773ee8d20364403ad609186320e117d" is_pre_election: true
I20260807 18:38:13.019346 10198 leader_election.cc:302] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74666dfde8e940509f5ebf53f1754017, e8a64bab799747d8a397357fd09181c8; no voters:
I20260807 18:38:13.019436 10069 raft_consensus.cc:2454] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 0.
I20260807 18:38:13.019665 10426 raft_consensus.cc:2781] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:13.019771 10426 raft_consensus.cc:482] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:13.019831 10426 raft_consensus.cc:3037] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:13.021109 10426 raft_consensus.cc:504] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } }
I20260807 18:38:13.021471 10426 leader_election.cc:288] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 election: Requested vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 4773ee8d20364403ad609186320e117d (127.6.138.66:41793)
W20260807 18:38:13.021796 9928 tablet.cc:2369] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:13.021878 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b151aa39f56f4a6f9751ea08a8bfa511" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4773ee8d20364403ad609186320e117d"
I20260807 18:38:13.021914 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b151aa39f56f4a6f9751ea08a8bfa511" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8"
I20260807 18:38:13.021996 10069 raft_consensus.cc:3037] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:13.022055 9826 raft_consensus.cc:3037] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:13.023306 10069 raft_consensus.cc:2454] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 1.
I20260807 18:38:13.023489 9826 raft_consensus.cc:2454] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 1.
I20260807 18:38:13.023787 10195 leader_election.cc:302] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4773ee8d20364403ad609186320e117d, 74666dfde8e940509f5ebf53f1754017; no voters:
I20260807 18:38:13.023994 10426 raft_consensus.cc:2781] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:13.024367 10426 raft_consensus.cc:686] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 1 LEADER]: Becoming Leader. State: Replica: 74666dfde8e940509f5ebf53f1754017, State: Running, Role: LEADER
I20260807 18:38:13.024739 10426 consensus_queue.cc:237] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } }
I20260807 18:38:13.028196 9589 catalog_manager.cc:5495] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74666dfde8e940509f5ebf53f1754017 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:38:13 dist-test-slave-2d19 krb5kdc[9528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127891, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:13.110671 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.102215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.6.138.126:42309)
0807 18:38:13.102382 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.102387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.102607 (+ 220us) server_negotiation.cc:407] Connection header received
0807 18:38:13.102773 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.102776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.102820 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.102906 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:13.103589 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.104177 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.105066 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.105305 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.105416 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.107681 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:13.107684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:13.107687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:13.107710 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:13.109061 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:13.109531 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:13.109533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:13.109535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:13.109615 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:13.109868 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:13.109870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:13.109872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:13.110047 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:13.110140 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.110416 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.110541 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":55}
I20260807 18:38:13.110670 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.102171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42309 (local address 127.0.0.1:53232)
0807 18:38:13.102508 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:13.102521 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:13.102671 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:13.102926 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:13.102929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:13.103368 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:13.103457 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.103463 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.104316 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.104319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.104940 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.104950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.105423 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.105426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.105525 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.106055 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:13.106067 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:13.107529 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:13.109185 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:13.109194 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:13.109199 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:13.109431 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:13.109700 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:13.109702 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:13.109703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:13.109785 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:13.110147 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:13.110153 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:13.110261 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 18:38:13.110354 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:38:13.110485 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260807 18:38:13.115091 9589 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:53232:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:38:13.115856 9589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:13.121225 10279 tablet_service.cc:1467] Processing CreateTablet for tablet 39c7225e2fc14d698ca7e63d580e7223 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1d38b4b34f4415ca0a2c158fc00a272]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:38:13.121084 10043 tablet_service.cc:1467] Processing CreateTablet for tablet 39c7225e2fc14d698ca7e63d580e7223 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1d38b4b34f4415ca0a2c158fc00a272]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:38:13.121061 9819 tablet_service.cc:1467] Processing CreateTablet for tablet 39c7225e2fc14d698ca7e63d580e7223 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e1d38b4b34f4415ca0a2c158fc00a272]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:38:13.121856 9819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39c7225e2fc14d698ca7e63d580e7223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:13.121856 10043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39c7225e2fc14d698ca7e63d580e7223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:13.121867 10279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39c7225e2fc14d698ca7e63d580e7223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:13.124887 10417 tablet_bootstrap.cc:492] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: Bootstrap starting.
I20260807 18:38:13.124961 10413 tablet_bootstrap.cc:492] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: Bootstrap starting.
I20260807 18:38:13.125399 10410 tablet_bootstrap.cc:492] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: Bootstrap starting.
I20260807 18:38:13.125897 10417 tablet_bootstrap.cc:654] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:13.126015 10413 tablet_bootstrap.cc:654] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:13.126408 10410 tablet_bootstrap.cc:654] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:13.127554 10417 tablet_bootstrap.cc:492] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: No bootstrap required, opened a new log
I20260807 18:38:13.127646 10417 ts_tablet_manager.cc:1397] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:13.127880 10413 tablet_bootstrap.cc:492] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: No bootstrap required, opened a new log
I20260807 18:38:13.127965 10413 ts_tablet_manager.cc:1397] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 18:38:13.128156 10417 raft_consensus.cc:348] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.128307 10417 raft_consensus.cc:374] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:13.128347 10417 raft_consensus.cc:729] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74666dfde8e940509f5ebf53f1754017, State: Initialized, Role: FOLLOWER
I20260807 18:38:13.128471 10417 consensus_queue.cc:260] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.128500 10413 raft_consensus.cc:348] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.128618 10413 raft_consensus.cc:374] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:13.128656 10413 raft_consensus.cc:729] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a64bab799747d8a397357fd09181c8, State: Initialized, Role: FOLLOWER
I20260807 18:38:13.128701 10417 ts_tablet_manager.cc:1428] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:13.128731 10426 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.128777 10413 consensus_queue.cc:260] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.128865 10426 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:13.129042 10413 ts_tablet_manager.cc:1428] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:13.129079 10425 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.129204 10425 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:13.129724 10410 tablet_bootstrap.cc:492] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: No bootstrap required, opened a new log
I20260807 18:38:13.129843 10410 ts_tablet_manager.cc:1397] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:38:13.130350 10410 raft_consensus.cc:348] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.130491 10410 raft_consensus.cc:374] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:13.130530 10410 raft_consensus.cc:729] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4773ee8d20364403ad609186320e117d, State: Initialized, Role: FOLLOWER
I20260807 18:38:13.130651 10410 consensus_queue.cc:260] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.130918 10410 ts_tablet_manager.cc:1428] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 18:38:13.131162 10424 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.131299 10424 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:13.159902 10426 raft_consensus.cc:482] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:13.160048 10426 raft_consensus.cc:504] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.160495 10426 leader_election.cc:288] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 4773ee8d20364403ad609186320e117d (127.6.138.66:41793)
I20260807 18:38:13.161015 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39c7225e2fc14d698ca7e63d580e7223" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8" is_pre_election: true
I20260807 18:38:13.161016 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39c7225e2fc14d698ca7e63d580e7223" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4773ee8d20364403ad609186320e117d" is_pre_election: true
I20260807 18:38:13.161190 9826 raft_consensus.cc:2454] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 0.
I20260807 18:38:13.161190 10069 raft_consensus.cc:2454] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 0.
I20260807 18:38:13.161510 10195 leader_election.cc:302] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4773ee8d20364403ad609186320e117d, 74666dfde8e940509f5ebf53f1754017; no voters:
I20260807 18:38:13.161723 10426 raft_consensus.cc:2781] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:13.161791 10426 raft_consensus.cc:482] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:13.161830 10426 raft_consensus.cc:3037] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:13.162973 10426 raft_consensus.cc:504] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.163331 10426 leader_election.cc:288] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 election: Requested vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 4773ee8d20364403ad609186320e117d (127.6.138.66:41793)
I20260807 18:38:13.163831 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39c7225e2fc14d698ca7e63d580e7223" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4773ee8d20364403ad609186320e117d"
I20260807 18:38:13.164000 10069 raft_consensus.cc:3037] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:13.163981 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39c7225e2fc14d698ca7e63d580e7223" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8"
I20260807 18:38:13.164115 9826 raft_consensus.cc:3037] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:13.165279 10069 raft_consensus.cc:2454] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 1.
I20260807 18:38:13.165280 9826 raft_consensus.cc:2454] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 1.
I20260807 18:38:13.165618 10198 leader_election.cc:302] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74666dfde8e940509f5ebf53f1754017, e8a64bab799747d8a397357fd09181c8; no voters:
I20260807 18:38:13.165812 10426 raft_consensus.cc:2781] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:13.165936 10426 raft_consensus.cc:686] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [term 1 LEADER]: Becoming Leader. State: Replica: 74666dfde8e940509f5ebf53f1754017, State: Running, Role: LEADER
I20260807 18:38:13.166077 10426 consensus_queue.cc:237] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.166795 10428 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: TxnStatusTablet state changed. Reason: New leader 74666dfde8e940509f5ebf53f1754017. Latest consensus state: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.166934 10428 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:38:13.167479 10445 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:38:13.167783 10445 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:38:13.167927 9591 catalog_manager.cc:5495] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74666dfde8e940509f5ebf53f1754017 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } health_report { overall_health: HEALTHY } } }
I20260807 18:38:13.186789 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.181977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:37691 (local address 127.0.0.1:53572)
0807 18:38:13.182169 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:13.182187 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:13.182298 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:13.182641 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:13.182643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:13.182650 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:13.182745 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.182749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.183983 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.183987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.184769 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.184779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.185445 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.185450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.185555 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.185623 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.186490 (+ 867us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.186509 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:38:13.186572 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:38:13.186660 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:38:13.186801 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.182146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53572 (local address 127.6.138.67:37691)
0807 18:38:13.182307 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.182315 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.182336 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:38:13.182425 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.182429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.182500 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.182623 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:13.182856 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.183854 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.184872 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.185269 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.185395 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.185816 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.186369 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.186497 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.186595 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.186662 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:38:13.192852 9826 raft_consensus.cc:1261] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Refusing update from remote peer 74666dfde8e940509f5ebf53f1754017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:13.192859 10069 raft_consensus.cc:1261] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Refusing update from remote peer 74666dfde8e940509f5ebf53f1754017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:13.193629 10428 consensus_queue.cc:1035] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:13.193835 10426 consensus_queue.cc:1035] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:13.197793 10424 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: TxnStatusTablet state changed. Reason: New leader 74666dfde8e940509f5ebf53f1754017. Latest consensus state: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.197937 10424 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:13.197940 10425 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: TxnStatusTablet state changed. Reason: New leader 74666dfde8e940509f5ebf53f1754017. Latest consensus state: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.198097 10425 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:13.200352 10449 mvcc.cc:205] Tried to move back new op lower bound from 7315979850510483456 to 7315979850409566208. Current Snapshot: MvccSnapshot[applied={T|T < 7315979850510483456}]
I20260807 18:38:13.200733 10428 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.200853 10428 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:38:13.200958 10428 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.201030 10428 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P 74666dfde8e940509f5ebf53f1754017: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:38:13.202646 10424 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.202754 10424 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P 4773ee8d20364403ad609186320e117d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:13.202867 10425 tablet_replica.cc:439] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } } }
I20260807 18:38:13.202997 10425 tablet_replica.cc:442] T 39c7225e2fc14d698ca7e63d580e7223 P e8a64bab799747d8a397357fd09181c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:13.205137 10452 mvcc.cc:205] Tried to move back new op lower bound from 7315979850510483456 to 7315979850409566208. Current Snapshot: MvccSnapshot[applied={T|T < 7315979850510483456}]
I20260807 18:38:13.222191 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.218614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.6.138.67:37691)
0807 18:38:13.218752 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.218759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.218810 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:38:13.218991 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.218994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.219046 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.219136 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:13.219416 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.219943 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.221049 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.221286 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.221400 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.221678 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.221760 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.221840 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.221982 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.222035 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:38:13.243848 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.239046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:37691 (local address 127.6.138.67:33131)
0807 18:38:13.239243 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:13.239260 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:13.239400 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:13.239720 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:13.239723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:13.239745 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:13.239908 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.239914 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.241157 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.241161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.242192 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.242203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.242880 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.242884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.242996 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.243060 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.243455 (+ 395us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.243462 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:38:13.243533 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 18:38:13.243667 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":49}
I20260807 18:38:13.243896 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.238172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:33131 (local address 127.6.138.67:37691)
0807 18:38:13.238300 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.238305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.239361 (+ 1056us) server_negotiation.cc:407] Connection header received
0807 18:38:13.239518 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.239523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.239597 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.239695 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:13.240048 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.241014 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.242319 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.242745 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.242878 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.243254 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.243358 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.243440 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.243545 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.243675 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":47}
I20260807 18:38:13.252657 9826 raft_consensus.cc:1261] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Refusing update from remote peer 74666dfde8e940509f5ebf53f1754017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:13.253062 10069 raft_consensus.cc:1261] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Refusing update from remote peer 74666dfde8e940509f5ebf53f1754017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:13.253515 10426 consensus_queue.cc:1035] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:13.253680 10428 consensus_queue.cc:1035] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:13.269505 10451 mvcc.cc:205] Tried to move back new op lower bound from 7315979850758012928 to 7315979849832382464. Current Snapshot: MvccSnapshot[applied={T|T < 7315979850758012928}]
I20260807 18:38:13.321043 9591 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:13.321246 9591 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:13.324883 9591 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: Sending DeleteTablet for 3 replicas of tablet b151aa39f56f4a6f9751ea08a8bfa511
I20260807 18:38:13.325937 10279 tablet_service.cc:1514] Processing DeleteTablet for tablet b151aa39f56f4a6f9751ea08a8bfa511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:13 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:53568
I20260807 18:38:13.325935 9819 tablet_service.cc:1514] Processing DeleteTablet for tablet b151aa39f56f4a6f9751ea08a8bfa511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:13 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:42298
I20260807 18:38:13.326498 10043 tablet_service.cc:1514] Processing DeleteTablet for tablet b151aa39f56f4a6f9751ea08a8bfa511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:13 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45754
I20260807 18:38:13.327082 10474 tablet_replica.cc:330] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: stopping tablet replica
I20260807 18:38:13.327399 9591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:38:13.327522 10474 raft_consensus.cc:2229] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:13.327769 10474 raft_consensus.cc:2258] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 18:38:13.327855 9591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:13.329010 10469 tablet_replica.cc:330] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: stopping tablet replica
I20260807 18:38:13.329382 10469 raft_consensus.cc:2229] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:13.329564 10474 ts_tablet_manager.cc:1905] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:13.329841 10470 tablet_replica.cc:330] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: stopping tablet replica
I20260807 18:38:13.329851 10469 raft_consensus.cc:2258] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:13.330210 10470 raft_consensus.cc:2229] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:13.331449 10470 raft_consensus.cc:2258] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:13.332290 10469 ts_tablet_manager.cc:1905] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:13.333936 9819 tablet_service.cc:1467] Processing CreateTablet for tablet 20139285b8ca4887a88e7e32f901c813 (DEFAULT_TABLE table=test-table [id=484ed0525d234cd1a7fa690690d697aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:13.334229 9819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20139285b8ca4887a88e7e32f901c813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:13.334196 10279 tablet_service.cc:1467] Processing CreateTablet for tablet 20139285b8ca4887a88e7e32f901c813 (DEFAULT_TABLE table=test-table [id=484ed0525d234cd1a7fa690690d697aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:13.334443 10279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20139285b8ca4887a88e7e32f901c813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:13.336776 10417 tablet_bootstrap.cc:492] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: Bootstrap starting.
I20260807 18:38:13.336943 10043 tablet_service.cc:1467] Processing CreateTablet for tablet 20139285b8ca4887a88e7e32f901c813 (DEFAULT_TABLE table=test-table [id=484ed0525d234cd1a7fa690690d697aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:13.337209 10043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20139285b8ca4887a88e7e32f901c813. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:13.337284 10469 ts_tablet_manager.cc:1918] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:13.337383 10469 log.cc:1196] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/b151aa39f56f4a6f9751ea08a8bfa511
I20260807 18:38:13.337785 10417 tablet_bootstrap.cc:654] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:13.337818 10469 ts_tablet_manager.cc:1939] T b151aa39f56f4a6f9751ea08a8bfa511 P 74666dfde8e940509f5ebf53f1754017: Deleting consensus metadata
I20260807 18:38:13.339946 9564 catalog_manager.cc:4841] TS 74666dfde8e940509f5ebf53f1754017 (127.6.138.67:37691): tablet b151aa39f56f4a6f9751ea08a8bfa511 (table test-table [id=116a45a52fd4410db3be27770a5ad68a]) successfully deleted
I20260807 18:38:13.340130 10413 tablet_bootstrap.cc:492] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: Bootstrap starting.
I20260807 18:38:13.341418 10413 tablet_bootstrap.cc:654] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:13.342801 10410 tablet_bootstrap.cc:492] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: Bootstrap starting.
I20260807 18:38:13.344327 10410 tablet_bootstrap.cc:654] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:13.346000 10410 tablet_bootstrap.cc:492] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: No bootstrap required, opened a new log
I20260807 18:38:13.346124 10410 ts_tablet_manager.cc:1397] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260807 18:38:13.346722 10410 raft_consensus.cc:348] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.346877 10410 raft_consensus.cc:374] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:13.346930 10410 raft_consensus.cc:729] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4773ee8d20364403ad609186320e117d, State: Initialized, Role: FOLLOWER
I20260807 18:38:13.347114 10410 consensus_queue.cc:260] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.347482 10410 ts_tablet_manager.cc:1428] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:38:13.351051 10474 ts_tablet_manager.cc:1918] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:13.351446 10470 ts_tablet_manager.cc:1905] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:13.355809 10470 ts_tablet_manager.cc:1918] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:13.355980 10470 log.cc:1196] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/b151aa39f56f4a6f9751ea08a8bfa511
I20260807 18:38:13.356559 10470 ts_tablet_manager.cc:1939] T b151aa39f56f4a6f9751ea08a8bfa511 P e8a64bab799747d8a397357fd09181c8: Deleting consensus metadata
I20260807 18:38:13.357933 10474 log.cc:1196] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/b151aa39f56f4a6f9751ea08a8bfa511
I20260807 18:38:13.358150 10413 tablet_bootstrap.cc:492] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: No bootstrap required, opened a new log
I20260807 18:38:13.358249 10413 ts_tablet_manager.cc:1397] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260807 18:38:13.358299 10417 tablet_bootstrap.cc:492] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: No bootstrap required, opened a new log
I20260807 18:38:13.358366 10474 ts_tablet_manager.cc:1939] T b151aa39f56f4a6f9751ea08a8bfa511 P 4773ee8d20364403ad609186320e117d: Deleting consensus metadata
I20260807 18:38:13.358388 10417 ts_tablet_manager.cc:1397] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20260807 18:38:13.358754 10413 raft_consensus.cc:348] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.358883 10413 raft_consensus.cc:374] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:13.358937 10413 raft_consensus.cc:729] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a64bab799747d8a397357fd09181c8, State: Initialized, Role: FOLLOWER
I20260807 18:38:13.358865 10417 raft_consensus.cc:348] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.358983 10417 raft_consensus.cc:374] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:13.359015 10417 raft_consensus.cc:729] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74666dfde8e940509f5ebf53f1754017, State: Initialized, Role: FOLLOWER
I20260807 18:38:13.359082 10413 consensus_queue.cc:260] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.359135 10417 consensus_queue.cc:260] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.359395 10413 ts_tablet_manager.cc:1428] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:13.359395 10417 ts_tablet_manager.cc:1428] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:13.359849 9564 catalog_manager.cc:4841] TS 4773ee8d20364403ad609186320e117d (127.6.138.66:41793): tablet b151aa39f56f4a6f9751ea08a8bfa511 (table test-table [id=116a45a52fd4410db3be27770a5ad68a]) successfully deleted
I20260807 18:38:13.361117 9567 catalog_manager.cc:4841] TS e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871): tablet b151aa39f56f4a6f9751ea08a8bfa511 (table test-table [id=116a45a52fd4410db3be27770a5ad68a]) successfully deleted
I20260807 18:38:13.657251 10424 raft_consensus.cc:482] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:13.657413 10424 raft_consensus.cc:504] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.658524 10424 leader_election.cc:288] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 74666dfde8e940509f5ebf53f1754017 (127.6.138.67:37691)
I20260807 18:38:13.662693 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.658814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:37691 (local address 127.6.138.66:55399)
0807 18:38:13.659185 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:13.659203 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:13.659355 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:13.659656 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:13.659662 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:13.659681 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:13.659827 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.659833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.660689 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.660693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.661585 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.661596 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.662241 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.662248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.662338 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.662361 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:38:13.662432 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 18:38:13.662508 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":81,"thread_start_us":82,"threads_started":1}
I20260807 18:38:13.662729 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.658908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:55399 (local address 127.6.138.67:37691)
0807 18:38:13.659062 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.659067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.659306 (+ 239us) server_negotiation.cc:407] Connection header received
0807 18:38:13.659486 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.659492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.659550 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.659617 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:13.659968 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.660555 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.661715 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.662115 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.662214 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.662289 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.662466 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.662560 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:38:13.662817 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.658897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:57731 (local address 127.6.138.65:41871)
0807 18:38:13.659188 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.659210 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.659366 (+ 156us) server_negotiation.cc:407] Connection header received
0807 18:38:13.659528 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.659530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.659572 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.659676 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:13.659979 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.660552 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.661757 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.662191 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.662311 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.662362 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.662585 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.662661 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":106,"threads_started":1}
I20260807 18:38:13.662743 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.658816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:41871 (local address 127.6.138.66:57731)
0807 18:38:13.659238 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:13.659250 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:13.659388 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:13.659705 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:13.659708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:13.659738 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:13.659851 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.659856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.660663 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.660668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.661597 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.661608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.662323 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.662327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.662445 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.662467 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:38:13.662543 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 18:38:13.662609 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":119,"spinlock_wait_cycles":896,"thread_start_us":87,"threads_started":1}
I20260807 18:38:13.663556 10307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20139285b8ca4887a88e7e32f901c813" candidate_uuid: "4773ee8d20364403ad609186320e117d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74666dfde8e940509f5ebf53f1754017" is_pre_election: true
I20260807 18:38:13.663524 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20139285b8ca4887a88e7e32f901c813" candidate_uuid: "4773ee8d20364403ad609186320e117d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8" is_pre_election: true
I20260807 18:38:13.663740 9826 raft_consensus.cc:2454] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4773ee8d20364403ad609186320e117d in term 0.
I20260807 18:38:13.663805 10307 raft_consensus.cc:2454] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4773ee8d20364403ad609186320e117d in term 0.
I20260807 18:38:13.664207 9978 leader_election.cc:302] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4773ee8d20364403ad609186320e117d, e8a64bab799747d8a397357fd09181c8; no voters:
I20260807 18:38:13.664588 10424 raft_consensus.cc:2781] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:13.664664 10424 raft_consensus.cc:482] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:13.664701 10424 raft_consensus.cc:3037] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:13.665742 10424 raft_consensus.cc:504] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.666093 10424 leader_election.cc:288] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [CANDIDATE]: Term 1 election: Requested vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 74666dfde8e940509f5ebf53f1754017 (127.6.138.67:37691)
I20260807 18:38:13.666476 10307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20139285b8ca4887a88e7e32f901c813" candidate_uuid: "4773ee8d20364403ad609186320e117d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74666dfde8e940509f5ebf53f1754017"
I20260807 18:38:13.666481 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20139285b8ca4887a88e7e32f901c813" candidate_uuid: "4773ee8d20364403ad609186320e117d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8"
I20260807 18:38:13.666628 9826 raft_consensus.cc:3037] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:13.666623 10307 raft_consensus.cc:3037] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:13.667862 9826 raft_consensus.cc:2454] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4773ee8d20364403ad609186320e117d in term 1.
I20260807 18:38:13.667925 10307 raft_consensus.cc:2454] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4773ee8d20364403ad609186320e117d in term 1.
I20260807 18:38:13.668175 9978 leader_election.cc:302] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4773ee8d20364403ad609186320e117d, e8a64bab799747d8a397357fd09181c8; no voters:
I20260807 18:38:13.668402 10424 raft_consensus.cc:2781] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:13.668635 10424 raft_consensus.cc:686] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 1 LEADER]: Becoming Leader. State: Replica: 4773ee8d20364403ad609186320e117d, State: Running, Role: LEADER
I20260807 18:38:13.668886 10424 consensus_queue.cc:237] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:13.671368 9583 catalog_manager.cc:5495] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4773ee8d20364403ad609186320e117d (127.6.138.66). New cstate: current_term: 1 leader_uuid: "4773ee8d20364403ad609186320e117d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } health_report { overall_health: UNKNOWN } } }
I20260807 18:38:13.710119 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.706498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.6.138.66:41793)
0807 18:38:13.706783 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.706790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.706802 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:38:13.706872 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.706873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.706923 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.707029 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:13.707232 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.707785 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.708628 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.708857 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.708937 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.709161 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.709635 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.709724 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.709859 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.709911 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":132,"threads_started":1}
I20260807 18:38:13.721309 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.717923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:57829 (local address 127.6.138.67:37691)
0807 18:38:13.718097 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.718101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.718237 (+ 136us) server_negotiation.cc:407] Connection header received
0807 18:38:13.718401 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.718404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.718455 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.718538 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:13.718798 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.719305 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.720174 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.720460 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.720571 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.720865 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.720936 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.721013 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.721108 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.721158 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:38:13.721339 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.717759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:37691 (local address 127.6.138.66:57829)
0807 18:38:13.718152 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:13.718164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:13.718293 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:13.718550 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:13.718554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:13.718561 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:13.718687 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.718692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.719421 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.719425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.720052 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.720060 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.720575 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.720579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.720677 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.720720 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.721030 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.721037 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:38:13.721097 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 18:38:13.721172 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":131,"threads_started":1}
I20260807 18:38:13.727432 9826 raft_consensus.cc:1261] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Refusing update from remote peer 4773ee8d20364403ad609186320e117d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:13.727663 10307 raft_consensus.cc:1261] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Refusing update from remote peer 4773ee8d20364403ad609186320e117d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:13.728291 10482 consensus_queue.cc:1035] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:13.728518 10424 consensus_queue.cc:1035] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [LEADER]: Connected to new peer: Peer: permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:13.750268 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.746576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:58211 (local address 127.6.138.66:41793)
0807 18:38:13.746678 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:13.746683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:13.746968 (+ 285us) server_negotiation.cc:407] Connection header received
0807 18:38:13.747120 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:13.747122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:13.747175 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:13.747257 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:13.747522 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.748035 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.749092 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.749420 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.749542 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.749821 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.749901 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.749943 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:13.750115 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:38:13.750168 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 18:38:13.750286 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:13.746495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:41793 (local address 127.6.138.67:58211)
0807 18:38:13.746870 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:13.746885 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:13.747013 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:13.747270 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:13.747273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:13.747281 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:13.747391 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.747396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.748160 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.748164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.748962 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:13.748973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:13.749548 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:13.749551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:13.749648 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:13.749680 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:13.750001 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:13.750006 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:38:13.750064 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 18:38:13.750142 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":58,"threads_started":1}
W20260807 18:38:13.751098 10451 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:38:15.321450 9591 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:15.321643 9591 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:15.324669 9591 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: Sending DeleteTablet for 3 replicas of tablet 20139285b8ca4887a88e7e32f901c813
I20260807 18:38:15.325485 10043 tablet_service.cc:1514] Processing DeleteTablet for tablet 20139285b8ca4887a88e7e32f901c813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:15 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45754
I20260807 18:38:15.325487 9819 tablet_service.cc:1514] Processing DeleteTablet for tablet 20139285b8ca4887a88e7e32f901c813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:15 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:42298
I20260807 18:38:15.325547 10279 tablet_service.cc:1514] Processing DeleteTablet for tablet 20139285b8ca4887a88e7e32f901c813 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:15 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:53568
I20260807 18:38:15.325917 10531 tablet_replica.cc:330] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: stopping tablet replica
I20260807 18:38:15.326076 10537 tablet_replica.cc:330] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: stopping tablet replica
I20260807 18:38:15.326160 10531 raft_consensus.cc:2229] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:15.326337 10537 raft_consensus.cc:2229] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:15.326512 10537 raft_consensus.cc:2258] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:15.326567 10531 raft_consensus.cc:2258] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:15.326963 10534 tablet_replica.cc:330] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: stopping tablet replica
I20260807 18:38:15.327127 10534 raft_consensus.cc:2229] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:15.327237 10534 raft_consensus.cc:2258] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:15.327798 10531 ts_tablet_manager.cc:1905] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:15.330188 10537 ts_tablet_manager.cc:1905] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:15.330492 10531 ts_tablet_manager.cc:1918] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:38:15.330590 10531 log.cc:1196] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/20139285b8ca4887a88e7e32f901c813
I20260807 18:38:15.330874 10531 ts_tablet_manager.cc:1939] T 20139285b8ca4887a88e7e32f901c813 P 4773ee8d20364403ad609186320e117d: Deleting consensus metadata
I20260807 18:38:15.331588 10534 ts_tablet_manager.cc:1905] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:15.331676 9564 catalog_manager.cc:4841] TS 4773ee8d20364403ad609186320e117d (127.6.138.66:41793): tablet 20139285b8ca4887a88e7e32f901c813 (table test-table [id=484ed0525d234cd1a7fa690690d697aa]) successfully deleted
I20260807 18:38:15.333141 10537 ts_tablet_manager.cc:1918] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:38:15.333294 10537 log.cc:1196] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/20139285b8ca4887a88e7e32f901c813
I20260807 18:38:15.333603 10537 ts_tablet_manager.cc:1939] T 20139285b8ca4887a88e7e32f901c813 P 74666dfde8e940509f5ebf53f1754017: Deleting consensus metadata
I20260807 18:38:15.334421 9564 catalog_manager.cc:4841] TS 74666dfde8e940509f5ebf53f1754017 (127.6.138.67:37691): tablet 20139285b8ca4887a88e7e32f901c813 (table test-table [id=484ed0525d234cd1a7fa690690d697aa]) successfully deleted
I20260807 18:38:15.334673 10534 ts_tablet_manager.cc:1918] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:38:15.334767 10534 log.cc:1196] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/20139285b8ca4887a88e7e32f901c813
I20260807 18:38:15.335068 10534 ts_tablet_manager.cc:1939] T 20139285b8ca4887a88e7e32f901c813 P e8a64bab799747d8a397357fd09181c8: Deleting consensus metadata
I20260807 18:38:15.335867 9567 catalog_manager.cc:4841] TS e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871): tablet 20139285b8ca4887a88e7e32f901c813 (table test-table [id=484ed0525d234cd1a7fa690690d697aa]) successfully deleted
Aug 07 18:38:15 dist-test-slave-2d19 krb5kdc[9528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127895, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:38:15 dist-test-slave-2d19 krb5kdc[9528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127895, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:15.357752 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:15.348772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53244 (local address 127.6.138.126:42309)
0807 18:38:15.349085 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:15.349093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:15.349250 (+ 157us) server_negotiation.cc:407] Connection header received
0807 18:38:15.349369 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:15.349372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:15.349419 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:15.349511 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:15.350264 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:15.350815 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:15.351616 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:15.351838 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:15.351929 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:15.354483 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:15.354487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:15.354489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:15.354513 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:15.355898 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:15.356439 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:15.356444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:15.356446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:15.356536 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:15.356826 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:15.356830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:15.356832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:15.357020 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:15.357177 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:15.357430 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 18:38:15.357563 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":104,"threads_started":1}
I20260807 18:38:15.360433 9591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:15.360862 9591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:15.365886 10043 tablet_service.cc:1467] Processing CreateTablet for tablet 2c849e069e084eb0a7d1e9afa2312ad5 (DEFAULT_TABLE table=test-table [id=7d774f902ed842bf81132abf76e7b802]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:15.365900 10279 tablet_service.cc:1467] Processing CreateTablet for tablet 2c849e069e084eb0a7d1e9afa2312ad5 (DEFAULT_TABLE table=test-table [id=7d774f902ed842bf81132abf76e7b802]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:15.365900 9819 tablet_service.cc:1467] Processing CreateTablet for tablet 2c849e069e084eb0a7d1e9afa2312ad5 (DEFAULT_TABLE table=test-table [id=7d774f902ed842bf81132abf76e7b802]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:15.366236 10279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c849e069e084eb0a7d1e9afa2312ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:15.366236 9819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c849e069e084eb0a7d1e9afa2312ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:15.366252 10043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c849e069e084eb0a7d1e9afa2312ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:15.369213 10554 tablet_bootstrap.cc:492] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: Bootstrap starting.
I20260807 18:38:15.369239 10553 tablet_bootstrap.cc:492] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: Bootstrap starting.
I20260807 18:38:15.369642 10551 tablet_bootstrap.cc:492] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: Bootstrap starting.
I20260807 18:38:15.370257 10554 tablet_bootstrap.cc:654] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:15.370268 10553 tablet_bootstrap.cc:654] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:15.370570 10551 tablet_bootstrap.cc:654] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:15.371776 10553 tablet_bootstrap.cc:492] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: No bootstrap required, opened a new log
I20260807 18:38:15.371805 10551 tablet_bootstrap.cc:492] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: No bootstrap required, opened a new log
I20260807 18:38:15.371843 10554 tablet_bootstrap.cc:492] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: No bootstrap required, opened a new log
I20260807 18:38:15.371897 10553 ts_tablet_manager.cc:1397] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:15.371934 10551 ts_tablet_manager.cc:1397] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:15.371991 10554 ts_tablet_manager.cc:1397] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:15.372345 10551 raft_consensus.cc:348] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.372455 10551 raft_consensus.cc:374] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:15.372486 10551 raft_consensus.cc:729] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74666dfde8e940509f5ebf53f1754017, State: Initialized, Role: FOLLOWER
I20260807 18:38:15.372417 10554 raft_consensus.cc:348] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.372417 10553 raft_consensus.cc:348] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.372527 10554 raft_consensus.cc:374] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:15.372527 10553 raft_consensus.cc:374] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:15.372562 10554 raft_consensus.cc:729] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4773ee8d20364403ad609186320e117d, State: Initialized, Role: FOLLOWER
I20260807 18:38:15.372562 10553 raft_consensus.cc:729] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a64bab799747d8a397357fd09181c8, State: Initialized, Role: FOLLOWER
I20260807 18:38:15.372618 10551 consensus_queue.cc:260] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.372694 10553 consensus_queue.cc:260] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.372694 10554 consensus_queue.cc:260] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.372911 10551 ts_tablet_manager.cc:1428] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:15.373080 10554 ts_tablet_manager.cc:1428] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:15.373080 10553 ts_tablet_manager.cc:1428] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:15.579821 10508 raft_consensus.cc:482] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:15.580017 10508 raft_consensus.cc:504] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.580533 10508 leader_election.cc:288] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 4773ee8d20364403ad609186320e117d (127.6.138.66:41793)
I20260807 18:38:15.581084 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c849e069e084eb0a7d1e9afa2312ad5" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8" is_pre_election: true
I20260807 18:38:15.581085 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c849e069e084eb0a7d1e9afa2312ad5" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4773ee8d20364403ad609186320e117d" is_pre_election: true
I20260807 18:38:15.581267 9826 raft_consensus.cc:2454] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 0.
I20260807 18:38:15.581278 10069 raft_consensus.cc:2454] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 0.
I20260807 18:38:15.581658 10195 leader_election.cc:302] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4773ee8d20364403ad609186320e117d, 74666dfde8e940509f5ebf53f1754017; no voters:
I20260807 18:38:15.581845 10508 raft_consensus.cc:2781] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:15.581907 10508 raft_consensus.cc:482] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:15.581934 10508 raft_consensus.cc:3037] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:15.583009 10508 raft_consensus.cc:504] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.583374 10508 leader_election.cc:288] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 election: Requested vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 4773ee8d20364403ad609186320e117d (127.6.138.66:41793)
I20260807 18:38:15.583807 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c849e069e084eb0a7d1e9afa2312ad5" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8"
I20260807 18:38:15.583822 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c849e069e084eb0a7d1e9afa2312ad5" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4773ee8d20364403ad609186320e117d"
I20260807 18:38:15.583947 9826 raft_consensus.cc:3037] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:15.583956 10069 raft_consensus.cc:3037] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:15.585168 9826 raft_consensus.cc:2454] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 1.
I20260807 18:38:15.585155 10069 raft_consensus.cc:2454] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 1.
I20260807 18:38:15.585575 10198 leader_election.cc:302] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74666dfde8e940509f5ebf53f1754017, e8a64bab799747d8a397357fd09181c8; no voters:
I20260807 18:38:15.585845 10508 raft_consensus.cc:2781] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:15.585976 10508 raft_consensus.cc:686] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 1 LEADER]: Becoming Leader. State: Replica: 74666dfde8e940509f5ebf53f1754017, State: Running, Role: LEADER
I20260807 18:38:15.586138 10508 consensus_queue.cc:237] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.588143 9589 catalog_manager.cc:5495] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74666dfde8e940509f5ebf53f1754017 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } health_report { overall_health: HEALTHY } } }
I20260807 18:38:15.614648 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:15.611344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53584 (local address 127.6.138.67:37691)
0807 18:38:15.611628 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:15.611636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:15.611648 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:38:15.611687 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:15.611689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:15.611730 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:15.611803 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:15.612126 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:15.612712 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:15.613605 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:15.613826 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:15.613908 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:15.614148 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:15.614220 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:15.614278 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:15.614385 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 18:38:15.614434 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":121,"threads_started":1}
I20260807 18:38:15.620931 9826 raft_consensus.cc:1261] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Refusing update from remote peer 74666dfde8e940509f5ebf53f1754017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:15.620983 10069 raft_consensus.cc:1261] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Refusing update from remote peer 74666dfde8e940509f5ebf53f1754017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:15.621410 10426 consensus_queue.cc:1035] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:15.621618 10501 consensus_queue.cc:1035] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:15.654052 9589 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:15.654204 9589 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:15.656410 9589 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: Sending DeleteTablet for 3 replicas of tablet 2c849e069e084eb0a7d1e9afa2312ad5
I20260807 18:38:15.657222 10043 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c849e069e084eb0a7d1e9afa2312ad5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:15 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45754
I20260807 18:38:15.657218 9819 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c849e069e084eb0a7d1e9afa2312ad5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:15 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:42298
I20260807 18:38:15.657415 10531 tablet_replica.cc:330] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: stopping tablet replica
I20260807 18:38:15.657553 10531 raft_consensus.cc:2229] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:15.657691 10531 raft_consensus.cc:2258] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:15.657734 10534 tablet_replica.cc:330] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: stopping tablet replica
I20260807 18:38:15.657738 10278 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c849e069e084eb0a7d1e9afa2312ad5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:15 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:53568
I20260807 18:38:15.657858 10534 raft_consensus.cc:2229] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:15.657796 9589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:38:15.657997 10534 raft_consensus.cc:2258] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:15.658016 10537 tablet_replica.cc:330] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: stopping tablet replica
I20260807 18:38:15.658134 10537 raft_consensus.cc:2229] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 1 LEADER]: Raft consensus shutting down.
W20260807 18:38:15.658236 9589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:15.658368 10537 raft_consensus.cc:2258] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:15.658829 10534 ts_tablet_manager.cc:1905] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:15.659029 10537 ts_tablet_manager.cc:1905] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:15.661576 10534 ts_tablet_manager.cc:1918] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:15.661656 10534 log.cc:1196] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/2c849e069e084eb0a7d1e9afa2312ad5
I20260807 18:38:15.661764 10537 ts_tablet_manager.cc:1918] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:15.661850 10537 log.cc:1196] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/2c849e069e084eb0a7d1e9afa2312ad5
I20260807 18:38:15.661901 10534 ts_tablet_manager.cc:1939] T 2c849e069e084eb0a7d1e9afa2312ad5 P e8a64bab799747d8a397357fd09181c8: Deleting consensus metadata
I20260807 18:38:15.662177 10537 ts_tablet_manager.cc:1939] T 2c849e069e084eb0a7d1e9afa2312ad5 P 74666dfde8e940509f5ebf53f1754017: Deleting consensus metadata
I20260807 18:38:15.662585 10531 ts_tablet_manager.cc:1905] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:15.662937 9564 catalog_manager.cc:4841] TS 74666dfde8e940509f5ebf53f1754017 (127.6.138.67:37691): tablet 2c849e069e084eb0a7d1e9afa2312ad5 (table test-table [id=7d774f902ed842bf81132abf76e7b802]) successfully deleted
I20260807 18:38:15.663300 9819 tablet_service.cc:1467] Processing CreateTablet for tablet a2d1e0a3b5724602bdb5e800842f4ed8 (DEFAULT_TABLE table=test-table [id=626683eefcfe4061be03d3aab11af11b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:15.663542 9819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2d1e0a3b5724602bdb5e800842f4ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:15.663777 9567 catalog_manager.cc:4841] TS e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871): tablet 2c849e069e084eb0a7d1e9afa2312ad5 (table test-table [id=7d774f902ed842bf81132abf76e7b802]) successfully deleted
I20260807 18:38:15.664973 10531 ts_tablet_manager.cc:1918] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:15.665028 10531 log.cc:1196] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/2c849e069e084eb0a7d1e9afa2312ad5
I20260807 18:38:15.665210 10531 ts_tablet_manager.cc:1939] T 2c849e069e084eb0a7d1e9afa2312ad5 P 4773ee8d20364403ad609186320e117d: Deleting consensus metadata
I20260807 18:38:15.665501 10553 tablet_bootstrap.cc:492] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8: Bootstrap starting.
I20260807 18:38:15.665545 10278 tablet_service.cc:1467] Processing CreateTablet for tablet a2d1e0a3b5724602bdb5e800842f4ed8 (DEFAULT_TABLE table=test-table [id=626683eefcfe4061be03d3aab11af11b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:15.665678 10043 tablet_service.cc:1467] Processing CreateTablet for tablet a2d1e0a3b5724602bdb5e800842f4ed8 (DEFAULT_TABLE table=test-table [id=626683eefcfe4061be03d3aab11af11b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:15.665795 10278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2d1e0a3b5724602bdb5e800842f4ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:15.665908 10043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2d1e0a3b5724602bdb5e800842f4ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:15.666461 10553 tablet_bootstrap.cc:654] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:15.667240 9564 catalog_manager.cc:4841] TS 4773ee8d20364403ad609186320e117d (127.6.138.66:41793): tablet 2c849e069e084eb0a7d1e9afa2312ad5 (table test-table [id=7d774f902ed842bf81132abf76e7b802]) successfully deleted
I20260807 18:38:15.667760 10553 tablet_bootstrap.cc:492] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8: No bootstrap required, opened a new log
I20260807 18:38:15.667837 10553 ts_tablet_manager.cc:1397] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:15.667845 10554 tablet_bootstrap.cc:492] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: Bootstrap starting.
I20260807 18:38:15.668895 10554 tablet_bootstrap.cc:654] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:15.669608 10551 tablet_bootstrap.cc:492] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: Bootstrap starting.
I20260807 18:38:15.670385 10551 tablet_bootstrap.cc:654] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:15.670990 10553 raft_consensus.cc:348] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.671132 10553 raft_consensus.cc:374] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:15.671181 10553 raft_consensus.cc:729] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a64bab799747d8a397357fd09181c8, State: Initialized, Role: FOLLOWER
I20260807 18:38:15.671329 10553 consensus_queue.cc:260] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.671490 10551 tablet_bootstrap.cc:492] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: No bootstrap required, opened a new log
I20260807 18:38:15.671587 10551 ts_tablet_manager.cc:1397] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:15.672041 10551 raft_consensus.cc:348] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.672183 10551 raft_consensus.cc:374] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:15.672190 10553 ts_tablet_manager.cc:1428] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:38:15.672231 10551 raft_consensus.cc:729] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74666dfde8e940509f5ebf53f1754017, State: Initialized, Role: FOLLOWER
I20260807 18:38:15.672390 10551 consensus_queue.cc:260] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.672672 10551 ts_tablet_manager.cc:1428] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:15.673076 10554 tablet_bootstrap.cc:492] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: No bootstrap required, opened a new log
I20260807 18:38:15.673187 10554 ts_tablet_manager.cc:1397] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 18:38:15.673612 10554 raft_consensus.cc:348] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.673748 10554 raft_consensus.cc:374] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:15.673803 10554 raft_consensus.cc:729] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4773ee8d20364403ad609186320e117d, State: Initialized, Role: FOLLOWER
I20260807 18:38:15.673940 10554 consensus_queue.cc:260] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.674261 10554 ts_tablet_manager.cc:1428] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:38:15.929821 10508 raft_consensus.cc:482] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:15.929986 10508 raft_consensus.cc:504] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.930567 10508 leader_election.cc:288] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 4773ee8d20364403ad609186320e117d (127.6.138.66:41793)
I20260807 18:38:15.931115 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2d1e0a3b5724602bdb5e800842f4ed8" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8" is_pre_election: true
I20260807 18:38:15.931151 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2d1e0a3b5724602bdb5e800842f4ed8" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4773ee8d20364403ad609186320e117d" is_pre_election: true
I20260807 18:38:15.931301 9850 raft_consensus.cc:2454] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 0.
I20260807 18:38:15.931315 10069 raft_consensus.cc:2454] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 0.
I20260807 18:38:15.931687 10195 leader_election.cc:302] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4773ee8d20364403ad609186320e117d, 74666dfde8e940509f5ebf53f1754017; no voters:
I20260807 18:38:15.931905 10508 raft_consensus.cc:2781] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:15.931967 10508 raft_consensus.cc:482] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:15.931995 10508 raft_consensus.cc:3037] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:15.933151 10508 raft_consensus.cc:504] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.933555 10508 leader_election.cc:288] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 election: Requested vote from peers e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871), 4773ee8d20364403ad609186320e117d (127.6.138.66:41793)
I20260807 18:38:15.933938 9850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2d1e0a3b5724602bdb5e800842f4ed8" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a64bab799747d8a397357fd09181c8"
I20260807 18:38:15.933951 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2d1e0a3b5724602bdb5e800842f4ed8" candidate_uuid: "74666dfde8e940509f5ebf53f1754017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4773ee8d20364403ad609186320e117d"
I20260807 18:38:15.934075 9850 raft_consensus.cc:3037] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:15.934150 10069 raft_consensus.cc:3037] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:15.935129 10069 raft_consensus.cc:2454] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 1.
I20260807 18:38:15.935144 9850 raft_consensus.cc:2454] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74666dfde8e940509f5ebf53f1754017 in term 1.
I20260807 18:38:15.935432 10195 leader_election.cc:302] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4773ee8d20364403ad609186320e117d, 74666dfde8e940509f5ebf53f1754017; no voters:
I20260807 18:38:15.935655 10508 raft_consensus.cc:2781] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:15.935806 10508 raft_consensus.cc:686] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 1 LEADER]: Becoming Leader. State: Replica: 74666dfde8e940509f5ebf53f1754017, State: Running, Role: LEADER
I20260807 18:38:15.935973 10508 consensus_queue.cc:237] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } }
I20260807 18:38:15.937772 9583 catalog_manager.cc:5495] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74666dfde8e940509f5ebf53f1754017 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "74666dfde8e940509f5ebf53f1754017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74666dfde8e940509f5ebf53f1754017" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 37691 } health_report { overall_health: HEALTHY } } }
I20260807 18:38:15.975198 9850 raft_consensus.cc:1261] T a2d1e0a3b5724602bdb5e800842f4ed8 P e8a64bab799747d8a397357fd09181c8 [term 1 FOLLOWER]: Refusing update from remote peer 74666dfde8e940509f5ebf53f1754017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:15.975307 10069 raft_consensus.cc:1261] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Refusing update from remote peer 74666dfde8e940509f5ebf53f1754017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:15.975642 10508 consensus_queue.cc:1035] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a64bab799747d8a397357fd09181c8" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 41871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:15.975845 10501 consensus_queue.cc:1035] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4773ee8d20364403ad609186320e117d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:38:15.989452 10497 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:38:17.539733 9591 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:17.539891 9591 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:17.542512 9591 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48b10794060949e5898c429e32d4c2d4: Sending DeleteTablet for 3 replicas of tablet a2d1e0a3b5724602bdb5e800842f4ed8
I20260807 18:38:17.543134 9819 tablet_service.cc:1514] Processing DeleteTablet for tablet a2d1e0a3b5724602bdb5e800842f4ed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:17 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:42298
I20260807 18:38:17.543238 10279 tablet_service.cc:1514] Processing DeleteTablet for tablet a2d1e0a3b5724602bdb5e800842f4ed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:17 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:53568
I20260807 18:38:17.543493 10043 tablet_service.cc:1514] Processing DeleteTablet for tablet a2d1e0a3b5724602bdb5e800842f4ed8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:17 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45754
I20260807 18:38:17.543561 10608 tablet_replica.cc:330] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: stopping tablet replica
I20260807 18:38:17.543653 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 9711
I20260807 18:38:17.543711 10608 raft_consensus.cc:2229] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:17.551971 10608 raft_consensus.cc:2258] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:17.553144 10608 ts_tablet_manager.cc:1905] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:17.556607 10610 tablet_replica.cc:330] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: stopping tablet replica
I20260807 18:38:17.556797 10610 raft_consensus.cc:2229] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:17.557044 10610 raft_consensus.cc:2258] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:17.557030 10608 ts_tablet_manager.cc:1918] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:38:17.557367 10608 log.cc:1196] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/a2d1e0a3b5724602bdb5e800842f4ed8
I20260807 18:38:17.557746 10608 ts_tablet_manager.cc:1939] T a2d1e0a3b5724602bdb5e800842f4ed8 P 74666dfde8e940509f5ebf53f1754017: Deleting consensus metadata
I20260807 18:38:17.557973 10610 ts_tablet_manager.cc:1905] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:17.558881 9564 catalog_manager.cc:4841] TS 74666dfde8e940509f5ebf53f1754017 (127.6.138.67:37691): tablet a2d1e0a3b5724602bdb5e800842f4ed8 (table test-table [id=626683eefcfe4061be03d3aab11af11b]) successfully deleted
I20260807 18:38:17.560408 10610 ts_tablet_manager.cc:1918] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:38:17.560504 10610 log.cc:1196] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/a2d1e0a3b5724602bdb5e800842f4ed8
I20260807 18:38:17.560758 10610 ts_tablet_manager.cc:1939] T a2d1e0a3b5724602bdb5e800842f4ed8 P 4773ee8d20364403ad609186320e117d: Deleting consensus metadata
I20260807 18:38:17.561518 9564 catalog_manager.cc:4841] TS 4773ee8d20364403ad609186320e117d (127.6.138.66:41793): tablet a2d1e0a3b5724602bdb5e800842f4ed8 (table test-table [id=626683eefcfe4061be03d3aab11af11b]) successfully deleted
W20260807 18:38:17.562247 9567 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:41871 (error 108)
I20260807 18:38:17.563146 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 9947
I20260807 18:38:17.563589 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:17.563071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:41871 (local address 127.0.0.1:56200)
0807 18:38:17.563372 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:17.563461 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:41871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":100,"threads_started":1}
W20260807 18:38:17.565676 9567 catalog_manager.cc:4584] TS e8a64bab799747d8a397357fd09181c8 (127.6.138.65:41871): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a2d1e0a3b5724602bdb5e800842f4ed8: Network error: Client connection negotiation failed: client connection to 127.6.138.65:41871: connect: Connection refused (error 111)
I20260807 18:38:17.578744 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 10172
I20260807 18:38:17.594450 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 9546
2026-08-07T18:38:17Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7256 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:17 dist-test-slave-2d19 krb5kdc[10617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:17 dist-test-slave-2d19 krb5kdc[10617](info): set up 2 sockets
Aug 07 18:38:17 dist-test-slave-2d19 krb5kdc[10617](info): commencing operation
krb5kdc: starting...
W20260807 18:38:18.880537 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:18 dist-test-slave-2d19 krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:19.020349 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:45787
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:45787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:19.138185 10636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:19.138463 10636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:19.138505 10636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:19.142635 10636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:19.142700 10636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:19.142720 10636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:19.142737 10636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:19.142752 10636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:19.148012 10636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:45787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:45787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:19.149298 10636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:19.150318 10636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:19.159792 10645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:19.159861 10636 server_base.cc:1034] running on GCE node
W20260807 18:38:19.159861 10643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:19.160303 10648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:19.160965 10636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:19.162134 10636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:19.163347 10636 hybrid_clock.cc:648] HybridClock initialized: now 1786127899163327 us; error 44 us; skew 500 ppm
Aug 07 18:38:19 dist-test-slave-2d19 krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127899, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:19.166386 10636 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:19.167356 10636 webserver.cc:460] Webserver started at http://127.6.138.126:44237/ using document root <none> and password file <none>
I20260807 18:38:19.168051 10636 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:19.168159 10636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:19.168445 10636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:19.170653 10636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "1d799b8a66da4c97933583ab28ca9c1e"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "f427be0238fc364bb3bc109a989b5cb6"
server_key_iv: "4ced580029a172e6801e784539dad6ca"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.171316 10636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "1d799b8a66da4c97933583ab28ca9c1e"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "f427be0238fc364bb3bc109a989b5cb6"
server_key_iv: "4ced580029a172e6801e784539dad6ca"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.175566 10636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:38:19.178391 10662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:19.179822 10636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:19.179984 10636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "1d799b8a66da4c97933583ab28ca9c1e"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "f427be0238fc364bb3bc109a989b5cb6"
server_key_iv: "4ced580029a172e6801e784539dad6ca"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.180127 10636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:19.207284 10636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:19.210888 10636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:19.211125 10636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:19.218299 10636 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:45787
I20260807 18:38:19.218379 10739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:45787 every 8 connection(s)
I20260807 18:38:19.219322 10636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:19.222261 10740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:19.227707 10740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e: Bootstrap starting.
I20260807 18:38:19.228331 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 10636
I20260807 18:38:19.228457 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:19.228722 6697 external_mini_cluster.cc:1428] Setting key de0d942812d61c6199963ab0b2b1769c
I20260807 18:38:19.230908 10740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:19.231979 10740 log.cc:824] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:19.234776 10740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e: No bootstrap required, opened a new log
I20260807 18:38:19.239997 10740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d799b8a66da4c97933583ab28ca9c1e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45787 } }
I20260807 18:38:19.240263 10740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:19.240358 10740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d799b8a66da4c97933583ab28ca9c1e, State: Initialized, Role: FOLLOWER
Aug 07 18:38:19 dist-test-slave-2d19 krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:19.240908 10740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d799b8a66da4c97933583ab28ca9c1e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45787 } }
I20260807 18:38:19.241076 10740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:19.241158 10740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:19.241267 10740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:19.242480 10740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d799b8a66da4c97933583ab28ca9c1e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45787 } }
I20260807 18:38:19.242878 10740 leader_election.cc:302] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d799b8a66da4c97933583ab28ca9c1e; no voters:
I20260807 18:38:19.243278 10740 leader_election.cc:288] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:19.243381 10746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:19.243607 10746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [term 1 LEADER]: Becoming Leader. State: Replica: 1d799b8a66da4c97933583ab28ca9c1e, State: Running, Role: LEADER
I20260807 18:38:19.244027 10746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d799b8a66da4c97933583ab28ca9c1e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45787 } }
I20260807 18:38:19.244742 10740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:19.245913 10747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d799b8a66da4c97933583ab28ca9c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d799b8a66da4c97933583ab28ca9c1e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45787 } } }
I20260807 18:38:19.246044 10747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:19.246522 10749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d799b8a66da4c97933583ab28ca9c1e. Latest consensus state: current_term: 1 leader_uuid: "1d799b8a66da4c97933583ab28ca9c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d799b8a66da4c97933583ab28ca9c1e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45787 } } }
I20260807 18:38:19.246692 10749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:19.247547 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:19.230337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57090 (local address 127.6.138.126:45787)
0807 18:38:19.230970 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:19.230996 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:19.231044 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:38:19.232161 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:19.232188 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:19.232742 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:19.233208 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:19.234436 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.236080 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.237562 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.238401 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.238607 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:19.241725 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:19.241730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:19.241743 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:19.241772 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:19.243922 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:19.244473 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:19.244479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:19.244486 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:19.244630 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:19.244976 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:19.244979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:19.244981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:19.245546 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:19.245784 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:19.246229 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 18:38:19.246567 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":402,"thread_start_us":164,"threads_started":1}
W20260807 18:38:19.249971 10766 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:38:19.250087 10766 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:38:19.250185 10757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:19.251284 10757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:19.257263 10757 catalog_manager.cc:1347] Generated new cluster ID: 631bbc79193d4bb5acb0bcc15ba5de88
I20260807 18:38:19.257331 10757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:19.276458 10757 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:19.277498 10757 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:19.288226 10757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d799b8a66da4c97933583ab28ca9c1e: Generated new TSK 0
I20260807 18:38:19.289175 10757 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:19.331960 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:45787
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:19.452493 10781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:19.452772 10781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:19.452817 10781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:19.456816 10781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:19.456879 10781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:19.456938 10781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:19.462302 10781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:45787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:19.463445 10781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:19.464526 10781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:19.474680 10791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:19.474675 10788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:19.474833 10781 server_base.cc:1034] running on GCE node
W20260807 18:38:19.474675 10789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:19.475420 10781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:19.476097 10781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:19.477317 10781 hybrid_clock.cc:648] HybridClock initialized: now 1786127899477300 us; error 29 us; skew 500 ppm
Aug 07 18:38:19 dist-test-slave-2d19 krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127899, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:19.480427 10781 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:19.481313 10781 webserver.cc:460] Webserver started at http://127.6.138.65:45229/ using document root <none> and password file <none>
I20260807 18:38:19.481899 10781 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:19.481994 10781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:19.482231 10781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:19.484491 10781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "f0eb39e19bbb47b2bc36235af75b3ddc"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "2f19a933a2ff8a114db067b6eebff38b"
server_key_iv: "d482d4f41f0aac59ef97b3f1f43b184b"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.485116 10781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "f0eb39e19bbb47b2bc36235af75b3ddc"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "2f19a933a2ff8a114db067b6eebff38b"
server_key_iv: "d482d4f41f0aac59ef97b3f1f43b184b"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.488855 10781 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:38:19.491318 10804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:19.492715 10781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:19.492853 10781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "f0eb39e19bbb47b2bc36235af75b3ddc"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "2f19a933a2ff8a114db067b6eebff38b"
server_key_iv: "d482d4f41f0aac59ef97b3f1f43b184b"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.492969 10781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:19.513095 10781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:19.516168 10781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:19.516409 10781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:19.517030 10781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:19.517889 10781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:19.517962 10781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:19.518034 10781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:19.518086 10781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:19.527235 10781 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:33459
I20260807 18:38:19.527261 10984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:33459 every 8 connection(s)
I20260807 18:38:19.528141 10781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:19.529959 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 10781
I20260807 18:38:19.530076 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:19.530411 6697 external_mini_cluster.cc:1428] Setting key 0533831988d5a03b679a4d9cc495d9a1
Aug 07 18:38:19 dist-test-slave-2d19 krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127899, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:38:19.543406 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:19.530082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:60409 (local address 127.6.138.126:45787)
0807 18:38:19.530268 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:19.530274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:19.531611 (+ 1337us) server_negotiation.cc:407] Connection header received
0807 18:38:19.532643 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:19.532649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:19.532713 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:19.532809 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:19.533993 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.534631 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.536221 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.536683 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.536813 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:19.539451 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:19.539455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:19.539460 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:19.539493 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:19.541287 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:19.541832 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:19.541839 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:19.541842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:19.541956 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:19.542332 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:19.542335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:19.542336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:19.542523 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:19.542609 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:19.543171 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 18:38:19.543281 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:38:19.544369 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:19.530501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:45787 (local address 127.6.138.65:60409)
0807 18:38:19.531435 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:19.531490 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:19.532421 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:19.533005 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:19.533012 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:19.533366 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:19.533836 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:19.533849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.534780 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.534786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:19.536076 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:19.536087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.536852 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.536855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:19.537003 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:19.537704 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:19.537721 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:19.539272 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:19.541435 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:19.541441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:19.541455 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:19.541714 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:19.542074 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:19.542076 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:19.542078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:19.542219 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:19.542627 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:19.542632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:19.542865 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 18:38:19.543027 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:38:19.543405 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":577,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
I20260807 18:38:19.545972 10987 heartbeater.cc:347] Connected to a master server at 127.6.138.126:45787
I20260807 18:38:19.546249 10987 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:19.546779 10987 heartbeater.cc:511] Master 127.6.138.126:45787 requested a full tablet report, sending...
I20260807 18:38:19.548535 10688 ts_manager.cc:194] Registered new tserver with Master: f0eb39e19bbb47b2bc36235af75b3ddc (127.6.138.65:33459)
I20260807 18:38:19.550192 10688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:60409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:19.563895 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:45787
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:19.682176 10994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:19.682439 10994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:19.682482 10994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:19.686723 10994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:19.686787 10994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:19.686846 10994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:19.692303 10994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:45787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:19.693354 10994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:19.694353 10994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:19.704654 11004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:19.704654 11000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:19.704752 11006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:19.704718 10994 server_base.cc:1034] running on GCE node
I20260807 18:38:19.705377 10994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:19.706072 10994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:19.707276 10994 hybrid_clock.cc:648] HybridClock initialized: now 1786127899707261 us; error 29 us; skew 500 ppm
Aug 07 18:38:19 dist-test-slave-2d19 krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127899, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:19.710156 10994 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:19.711097 10994 webserver.cc:460] Webserver started at http://127.6.138.66:43117/ using document root <none> and password file <none>
I20260807 18:38:19.711711 10994 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:19.711817 10994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:19.712095 10994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:19.714249 10994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "b4977aa6e9724e298c12c0b4d999696a"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "48920fe7af8f3374c79e18cc2c097ba9"
server_key_iv: "ab43c95961d904b74d783eaf309879ee"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.714892 10994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "b4977aa6e9724e298c12c0b4d999696a"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "48920fe7af8f3374c79e18cc2c097ba9"
server_key_iv: "ab43c95961d904b74d783eaf309879ee"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.718796 10994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 18:38:19.721359 11020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:19.722766 10994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:19.722942 10994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "b4977aa6e9724e298c12c0b4d999696a"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "48920fe7af8f3374c79e18cc2c097ba9"
server_key_iv: "ab43c95961d904b74d783eaf309879ee"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.723079 10994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:19.735684 10994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:19.738915 10994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:19.739156 10994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:19.739800 10994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:19.740841 10994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:19.740936 10994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:19.741019 10994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:19.741060 10994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:19.751655 10994 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:38361
I20260807 18:38:19.751677 11199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:38361 every 8 connection(s)
I20260807 18:38:19.752755 10994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:19.761365 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 10994
I20260807 18:38:19.761518 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:19.761809 6697 external_mini_cluster.cc:1428] Setting key 62b825cd85a5195eedb432e606235183
Aug 07 18:38:19 dist-test-slave-2d19 krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127899, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:19.767364 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:19.754720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:35713 (local address 127.6.138.126:45787)
0807 18:38:19.754910 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:19.754915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:19.756022 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 18:38:19.757050 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:19.757053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:19.757101 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:19.757168 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:19.758373 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.758909 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.760021 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.760226 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.760367 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:19.763157 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:19.763163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:19.763169 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:19.763195 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:19.764781 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:19.765457 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:19.765461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:19.765465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:19.765589 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:19.765995 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:19.766000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:19.766003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:19.766266 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:19.766376 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:19.766976 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 18:38:19.767122 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:38:19.768301 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:19.755137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:45787 (local address 127.6.138.66:35713)
0807 18:38:19.755834 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:19.755876 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:19.756828 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:19.757341 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:19.757347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:19.757709 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:19.758231 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:19.758245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.759040 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.759043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:19.759906 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:19.759914 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:19.760418 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:19.760421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:19.760567 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:19.761245 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:19.761262 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:19.762989 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:19.764907 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:19.764917 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:19.764936 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:19.765333 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:19.765705 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:19.765707 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:19.765709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:19.765888 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:19.766417 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:19.766423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:19.766699 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 18:38:19.766888 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 18:38:19.767230 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":123,"threads_started":1}
I20260807 18:38:19.769747 11200 heartbeater.cc:347] Connected to a master server at 127.6.138.126:45787
I20260807 18:38:19.770038 11200 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:19.770589 11200 heartbeater.cc:511] Master 127.6.138.126:45787 requested a full tablet report, sending...
I20260807 18:38:19.771847 10688 ts_manager.cc:194] Registered new tserver with Master: b4977aa6e9724e298c12c0b4d999696a (127.6.138.66:38361)
I20260807 18:38:19.772799 10688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:35713
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:19.796633 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:45787
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:19.917933 11205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:19.918372 11205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:19.918479 11205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:19.926184 11205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:19.926288 11205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:19.926388 11205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:19.935667 11205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:45787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:19.937551 11205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:19.940349 11205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:19.956405 11214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:19.956557 11217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:19.956404 11210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:19.956970 11205 server_base.cc:1034] running on GCE node
I20260807 18:38:19.957401 11205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:19.958283 11205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:19.959457 11205 hybrid_clock.cc:648] HybridClock initialized: now 1786127899959434 us; error 65 us; skew 500 ppm
Aug 07 18:38:19 dist-test-slave-2d19 krb5kdc[10617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127899, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:19.962399 11205 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:19.963565 11205 webserver.cc:460] Webserver started at http://127.6.138.67:44001/ using document root <none> and password file <none>
I20260807 18:38:19.964342 11205 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:19.964433 11205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:19.964762 11205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:19.967180 11205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "69112dda738f45d190fc2ba758c936e1"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "d9cf457235e80969bfc4aaa215b78203"
server_key_iv: "00c07451c6a58310c544764b7770d3de"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.967866 11205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "69112dda738f45d190fc2ba758c936e1"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "d9cf457235e80969bfc4aaa215b78203"
server_key_iv: "00c07451c6a58310c544764b7770d3de"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.972530 11205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260807 18:38:19.975405 11226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:19.977222 11205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:19.977382 11205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "69112dda738f45d190fc2ba758c936e1"
format_stamp: "Formatted at 2026-08-07 18:38:19 on dist-test-slave-2d19"
server_key: "d9cf457235e80969bfc4aaa215b78203"
server_key_iv: "00c07451c6a58310c544764b7770d3de"
server_key_version: "encryptionkey@0"
I20260807 18:38:19.977540 11205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:19.990381 11205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:19.993849 11205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:19.994078 11205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:19.994735 11205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:19.995618 11205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:19.995695 11205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:19.995769 11205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:19.995821 11205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:20.006215 11205 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:40241
I20260807 18:38:20.006238 11422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:40241 every 8 connection(s)
I20260807 18:38:20.007190 11205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:20.008816 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 11205
I20260807 18:38:20.008960 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:20.009311 6697 external_mini_cluster.cc:1428] Setting key f3e56f581fc2234395ee80883f9da829
Aug 07 18:38:20 dist-test-slave-2d19 krb5kdc[10617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127899, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:20.021201 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:20.009139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:40819 (local address 127.6.138.126:45787)
0807 18:38:20.009312 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:20.009318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:20.010591 (+ 1273us) server_negotiation.cc:407] Connection header received
0807 18:38:20.011699 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:20.011702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:20.011756 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:20.011834 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:20.013140 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:20.013676 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:20.014861 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:20.015094 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:20.015216 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:20.017777 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:20.017780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:20.017782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:20.017808 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:20.019184 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:20.019707 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:20.019710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:20.019712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:20.019792 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:20.020120 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:20.020122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:20.020123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:20.020372 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:20.020454 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:20.020957 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 18:38:20.021074 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:38:20.022076 11426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:20.009580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:45787 (local address 127.6.138.67:40819)
0807 18:38:20.010438 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:20.010495 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:20.011465 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:20.012030 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:20.012038 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:20.012508 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:20.012991 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:20.013004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:20.013818 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:20.013827 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:20.014723 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:20.014732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:20.015260 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:20.015264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:20.015438 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:20.016019 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:20.016036 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:20.017594 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:20.019298 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:20.019303 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:20.019315 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:20.019599 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:20.019883 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:20.019885 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:20.019887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:20.020029 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:20.020469 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:20.020474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:20.020733 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 18:38:20.020903 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 18:38:20.021176 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":167,"threads_started":1}
I20260807 18:38:20.023355 11424 heartbeater.cc:347] Connected to a master server at 127.6.138.126:45787
I20260807 18:38:20.023633 11424 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:20.024166 11424 heartbeater.cc:511] Master 127.6.138.126:45787 requested a full tablet report, sending...
I20260807 18:38:20.025264 10687 ts_manager.cc:194] Registered new tserver with Master: 69112dda738f45d190fc2ba758c936e1 (127.6.138.67:40241)
I20260807 18:38:20.025807 10687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:40819
I20260807 18:38:20.035648 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:38:20.053303 10744 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:20.051547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.6.138.126:45787)
0807 18:38:20.051742 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:20.051747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:20.051927 (+ 180us) server_negotiation.cc:407] Connection header received
0807 18:38:20.052122 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:20.052126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:20.052178 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:20.052297 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:20.053125 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57092: BlockingRecv error: recv got EOF from 127.0.0.1:57092 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:38:20.055146 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 10781
I20260807 18:38:20.068028 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 10994
I20260807 18:38:20.085052 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 11205
I20260807 18:38:20.096043 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 10636
2026-08-07T18:38:20Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2494 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:20 dist-test-slave-2d19 krb5kdc[11440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:20 dist-test-slave-2d19 krb5kdc[11440](info): set up 2 sockets
Aug 07 18:38:20 dist-test-slave-2d19 krb5kdc[11440](info): commencing operation
krb5kdc: starting...
W20260807 18:38:21.404481 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:21 dist-test-slave-2d19 krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:21.543488 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42091
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:21.659803 11466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:21.660148 11466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:21.660225 11466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:21.664314 11466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:21.664417 11466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:21.664455 11466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:21.664492 11466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:21.664523 11466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:21.669924 11466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:21.671082 11466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:21.672195 11466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:21.681123 11477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:21.681133 11473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:21.681144 11474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:21.681425 11466 server_base.cc:1034] running on GCE node
I20260807 18:38:21.682070 11466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:21.683255 11466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:21.684422 11466 hybrid_clock.cc:648] HybridClock initialized: now 1786127901684390 us; error 44 us; skew 500 ppm
Aug 07 18:38:21 dist-test-slave-2d19 krb5kdc[11440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127901, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:21.687120 11466 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:21.687965 11466 webserver.cc:460] Webserver started at http://127.6.138.126:45407/ using document root <none> and password file <none>
I20260807 18:38:21.688576 11466 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:21.688649 11466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:21.688827 11466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:21.690949 11466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "cba918893a7b4cbebf5049e2cff49bfd"
format_stamp: "Formatted at 2026-08-07 18:38:21 on dist-test-slave-2d19"
server_key: "48402ba7635c64140bd6f6a0638a959d"
server_key_iv: "bfb48732c8c1f1b17bca5300b9643696"
server_key_version: "encryptionkey@0"
I20260807 18:38:21.691469 11466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "cba918893a7b4cbebf5049e2cff49bfd"
format_stamp: "Formatted at 2026-08-07 18:38:21 on dist-test-slave-2d19"
server_key: "48402ba7635c64140bd6f6a0638a959d"
server_key_iv: "bfb48732c8c1f1b17bca5300b9643696"
server_key_version: "encryptionkey@0"
I20260807 18:38:21.695343 11466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:38:21.697919 11492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:21.699234 11466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:21.699335 11466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "cba918893a7b4cbebf5049e2cff49bfd"
format_stamp: "Formatted at 2026-08-07 18:38:21 on dist-test-slave-2d19"
server_key: "48402ba7635c64140bd6f6a0638a959d"
server_key_iv: "bfb48732c8c1f1b17bca5300b9643696"
server_key_version: "encryptionkey@0"
I20260807 18:38:21.699421 11466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:21.721391 11466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:21.724969 11466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:21.725134 11466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:21.732468 11466 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:42091
I20260807 18:38:21.732789 11579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:42091 every 8 connection(s)
I20260807 18:38:21.733479 11466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:21.736415 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:21.741215 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 11466
I20260807 18:38:21.741315 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:21.741611 6697 external_mini_cluster.cc:1428] Setting key 626a018d49764e3e21fcdc8a49a0bfb7
I20260807 18:38:21.742158 11580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd: Bootstrap starting.
I20260807 18:38:21.744871 11580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:21.745805 11580 log.cc:824] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:21.748006 11580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd: No bootstrap required, opened a new log
Aug 07 18:38:21 dist-test-slave-2d19 krb5kdc[11440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:21.752018 11580 raft_consensus.cc:348] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba918893a7b4cbebf5049e2cff49bfd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42091 } }
I20260807 18:38:21.752344 11580 raft_consensus.cc:374] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:21.752444 11580 raft_consensus.cc:729] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba918893a7b4cbebf5049e2cff49bfd, State: Initialized, Role: FOLLOWER
I20260807 18:38:21.752952 11580 consensus_queue.cc:260] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba918893a7b4cbebf5049e2cff49bfd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42091 } }
I20260807 18:38:21.753114 11580 raft_consensus.cc:388] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:21.753189 11580 raft_consensus.cc:482] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:21.753291 11580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:21.754204 11580 raft_consensus.cc:504] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba918893a7b4cbebf5049e2cff49bfd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42091 } }
I20260807 18:38:21.754565 11580 leader_election.cc:302] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cba918893a7b4cbebf5049e2cff49bfd; no voters:
I20260807 18:38:21.754925 11580 leader_election.cc:288] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:21.755004 11587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:21.755244 11587 raft_consensus.cc:686] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [term 1 LEADER]: Becoming Leader. State: Replica: cba918893a7b4cbebf5049e2cff49bfd, State: Running, Role: LEADER
I20260807 18:38:21.755558 11587 consensus_queue.cc:237] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba918893a7b4cbebf5049e2cff49bfd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42091 } }
I20260807 18:38:21.756318 11580 sys_catalog.cc:564] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:21.756791 11587 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cba918893a7b4cbebf5049e2cff49bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba918893a7b4cbebf5049e2cff49bfd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42091 } } }
I20260807 18:38:21.756907 11587 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:21.757232 11588 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader cba918893a7b4cbebf5049e2cff49bfd. Latest consensus state: current_term: 1 leader_uuid: "cba918893a7b4cbebf5049e2cff49bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba918893a7b4cbebf5049e2cff49bfd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42091 } } }
I20260807 18:38:21.757319 11588 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:21.757372 11593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:21.757795 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:21.743069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.6.138.126:42091)
0807 18:38:21.743523 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:21.743543 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:21.743594 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:38:21.744432 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:21.744456 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:21.744894 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:21.745281 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:21.746006 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:21.747369 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:21.748391 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:21.749074 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:21.749259 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:21.751736 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:21.751743 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:21.751759 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:21.751789 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:21.753921 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:21.754369 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:21.754375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:21.754381 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:21.754514 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:21.754844 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:21.754848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:21.754850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:21.755393 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:21.756194 (+ 801us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:21.756416 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 18:38:21.756735 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":104,"threads_started":1}
I20260807 18:38:21.760740 11593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:21.761143 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:38:21.766645 11593 catalog_manager.cc:1347] Generated new cluster ID: 4c6ad5b6607e429cac63a918a8aefcf8
I20260807 18:38:21.766726 11593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:21.780850 11593 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:21.782065 11593 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:21.785549 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:21.779379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57438 (local address 127.6.138.126:42091)
0807 18:38:21.779493 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:21.779498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:21.779726 (+ 228us) server_negotiation.cc:407] Connection header received
0807 18:38:21.779987 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:21.779990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:21.780048 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:21.780145 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:21.780944 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:21.781690 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:21.782988 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:21.783350 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:21.783458 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:21.784468 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:21.784472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:21.784475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:21.785055 (+ 580us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:21.785089 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:21.785108 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:21.785192 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:21.785330 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 18:38:21.785408 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 18:38:21.791437 11593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cba918893a7b4cbebf5049e2cff49bfd: Generated new TSK 0
I20260807 18:38:21.792187 11593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:38:21.800096 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 11466
2026-08-07T18:38:21Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1711 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:21 dist-test-slave-2d19 krb5kdc[11631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:21 dist-test-slave-2d19 krb5kdc[11631](info): set up 2 sockets
Aug 07 18:38:21 dist-test-slave-2d19 krb5kdc[11631](info): commencing operation
krb5kdc: starting...
W20260807 18:38:23.080524 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:23 dist-test-slave-2d19 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:23.224552 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:44397
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:44397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:23.344148 11647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:23.344552 11647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:23.344640 11647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:23.348868 11647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:23.348955 11647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:23.348992 11647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:23.349030 11647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:23.349068 11647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:23.354605 11647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:44397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:44397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:23.355751 11647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:23.356846 11647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:23.365823 11659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:23.365823 11656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:23.365828 11655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:23.366026 11647 server_base.cc:1034] running on GCE node
I20260807 18:38:23.366801 11647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:23.367993 11647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:23.369187 11647 hybrid_clock.cc:648] HybridClock initialized: now 1786127903369158 us; error 41 us; skew 500 ppm
Aug 07 18:38:23 dist-test-slave-2d19 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:23.380395 11647 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:23.381493 11647 webserver.cc:460] Webserver started at http://127.6.138.126:35481/ using document root <none> and password file <none>
I20260807 18:38:23.382133 11647 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:23.382236 11647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:23.382498 11647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:23.384773 11647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "fd7dffe307594a1aaa96f03753172cf1"
format_stamp: "Formatted at 2026-08-07 18:38:23 on dist-test-slave-2d19"
server_key: "822521b7ed7665d229501ed597f1607d"
server_key_iv: "4e060d85eb1eb70e897c88d4c7735e40"
server_key_version: "encryptionkey@0"
I20260807 18:38:23.385378 11647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "fd7dffe307594a1aaa96f03753172cf1"
format_stamp: "Formatted at 2026-08-07 18:38:23 on dist-test-slave-2d19"
server_key: "822521b7ed7665d229501ed597f1607d"
server_key_iv: "4e060d85eb1eb70e897c88d4c7735e40"
server_key_version: "encryptionkey@0"
I20260807 18:38:23.389348 11647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:38:23.392009 11670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:23.393395 11647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:38:23.393544 11647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "fd7dffe307594a1aaa96f03753172cf1"
format_stamp: "Formatted at 2026-08-07 18:38:23 on dist-test-slave-2d19"
server_key: "822521b7ed7665d229501ed597f1607d"
server_key_iv: "4e060d85eb1eb70e897c88d4c7735e40"
server_key_version: "encryptionkey@0"
I20260807 18:38:23.393651 11647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:23.444948 11647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:23.448277 11647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:23.448597 11647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:23.456430 11647 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:44397
I20260807 18:38:23.456810 11747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:44397 every 8 connection(s)
I20260807 18:38:23.457500 11647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:23.460474 11748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:23.463088 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 11647
I20260807 18:38:23.463240 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:23.463527 6697 external_mini_cluster.cc:1428] Setting key a80f0b9dc75c4ff8037a34ffbddb4a57
I20260807 18:38:23.466682 11748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1: Bootstrap starting.
I20260807 18:38:23.469462 11748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:23.470212 11748 log.cc:824] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:23.472424 11748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1: No bootstrap required, opened a new log
Aug 07 18:38:23 dist-test-slave-2d19 krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:23.476790 11748 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7dffe307594a1aaa96f03753172cf1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44397 } }
I20260807 18:38:23.477093 11748 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:23.477185 11748 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd7dffe307594a1aaa96f03753172cf1, State: Initialized, Role: FOLLOWER
I20260807 18:38:23.477715 11748 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7dffe307594a1aaa96f03753172cf1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44397 } }
I20260807 18:38:23.477912 11748 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:23.477990 11748 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:23.478102 11748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:23.479065 11748 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7dffe307594a1aaa96f03753172cf1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44397 } }
I20260807 18:38:23.479470 11748 leader_election.cc:302] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd7dffe307594a1aaa96f03753172cf1; no voters:
I20260807 18:38:23.479863 11748 leader_election.cc:288] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:23.479919 11755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:23.480129 11755 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [term 1 LEADER]: Becoming Leader. State: Replica: fd7dffe307594a1aaa96f03753172cf1, State: Running, Role: LEADER
I20260807 18:38:23.480482 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:23.465048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42346 (local address 127.6.138.126:44397)
0807 18:38:23.465493 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:23.465514 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:23.465557 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:38:23.466350 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:23.466373 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:23.466787 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:23.467176 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:23.467872 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:23.469438 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:23.470343 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:23.470919 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:23.471118 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:23.473988 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:23.473994 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:23.474012 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:23.474052 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:23.476889 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:23.477464 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:23.477470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:23.477475 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:23.477616 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:23.478031 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:23.478035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:23.478037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:23.478619 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:23.478866 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:23.479166 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:38:23.479470 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":109,"threads_started":1}
I20260807 18:38:23.480476 11755 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7dffe307594a1aaa96f03753172cf1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44397 } }
I20260807 18:38:23.481205 11748 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:23.483220 11756 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd7dffe307594a1aaa96f03753172cf1. Latest consensus state: current_term: 1 leader_uuid: "fd7dffe307594a1aaa96f03753172cf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7dffe307594a1aaa96f03753172cf1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44397 } } }
I20260807 18:38:23.483697 11756 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:23.483736 11758 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd7dffe307594a1aaa96f03753172cf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd7dffe307594a1aaa96f03753172cf1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44397 } } }
I20260807 18:38:23.483862 11758 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1 [sys.catalog]: This master's current role is: LEADER
W20260807 18:38:23.486711 11780 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:38:23.486779 11780 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:38:23.486860 11785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:23.487586 11785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:23.493603 11785 catalog_manager.cc:1347] Generated new cluster ID: 2cb26ed01c834b3aa1e70b6026c5ed35
I20260807 18:38:23.493682 11785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:23.505028 11785 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:23.506039 11785 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:23.511843 11785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd7dffe307594a1aaa96f03753172cf1: Generated new TSK 0
I20260807 18:38:23.512702 11785 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:23.551836 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44397
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:23.678566 11793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:23.678851 11793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:23.678897 11793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:23.683288 11793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:23.683360 11793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:23.683455 11793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:23.689458 11793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:23.690691 11793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:23.691813 11793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:23.703163 11803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:23.703315 11800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:23.703131 11801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:23.703502 11793 server_base.cc:1034] running on GCE node
I20260807 18:38:23.704077 11793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:23.704803 11793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:23.706023 11793 hybrid_clock.cc:648] HybridClock initialized: now 1786127903705988 us; error 46 us; skew 500 ppm
Aug 07 18:38:23 dist-test-slave-2d19 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:23.709255 11793 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:23.710206 11793 webserver.cc:460] Webserver started at http://127.6.138.65:33819/ using document root <none> and password file <none>
I20260807 18:38:23.710811 11793 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:23.710876 11793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:23.711120 11793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:23.713430 11793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "5dca27cc4da64ae8aa10afbeab0e2ea3"
format_stamp: "Formatted at 2026-08-07 18:38:23 on dist-test-slave-2d19"
server_key: "9df96cb6061c056736e4bb54c9ec293f"
server_key_iv: "da101e6e27842b97a294a431a590dc1f"
server_key_version: "encryptionkey@0"
I20260807 18:38:23.714104 11793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "5dca27cc4da64ae8aa10afbeab0e2ea3"
format_stamp: "Formatted at 2026-08-07 18:38:23 on dist-test-slave-2d19"
server_key: "9df96cb6061c056736e4bb54c9ec293f"
server_key_iv: "da101e6e27842b97a294a431a590dc1f"
server_key_version: "encryptionkey@0"
I20260807 18:38:23.718040 11793 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 18:38:23.720659 11812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:23.721942 11793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:23.722098 11793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "5dca27cc4da64ae8aa10afbeab0e2ea3"
format_stamp: "Formatted at 2026-08-07 18:38:23 on dist-test-slave-2d19"
server_key: "9df96cb6061c056736e4bb54c9ec293f"
server_key_iv: "da101e6e27842b97a294a431a590dc1f"
server_key_version: "encryptionkey@0"
I20260807 18:38:23.722211 11793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:23.742491 11793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:23.745682 11793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:23.746127 11793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:23.746963 11793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:23.747983 11793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:23.748068 11793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:23.748121 11793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:23.748138 11793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:23.758867 11793 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:41777
I20260807 18:38:23.758894 12006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:41777 every 8 connection(s)
I20260807 18:38:23.759907 11793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:23.760871 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 11793
I20260807 18:38:23.761026 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:23.761366 6697 external_mini_cluster.cc:1428] Setting key b7d3469c2c362f4d1cce917ee3c60315
Aug 07 18:38:23 dist-test-slave-2d19 krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
I20260807 18:38:23.777830 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:23.762031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:42501 (local address 127.6.138.126:44397)
0807 18:38:23.762201 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:23.762205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:23.763496 (+ 1291us) server_negotiation.cc:407] Connection header received
0807 18:38:23.765605 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:23.765610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:23.765673 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:23.765775 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:23.767031 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:23.767607 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:23.769646 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:23.769889 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:23.769982 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:23.773625 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:23.773629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:23.773632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:23.773661 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:23.775326 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:23.775919 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:23.775923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:23.775924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:23.776008 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:23.776538 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:23.776542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:23.776544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:23.776745 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:23.776868 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:23.777506 (+ 638us) server_negotiation.cc:299] Negotiation successful
0807 18:38:23.777648 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:38:23.778703 12010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:23.762694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44397 (local address 127.6.138.65:42501)
0807 18:38:23.763285 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:23.763351 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:23.765350 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:23.766006 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:23.766013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:23.766380 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:23.766884 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:23.766899 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:23.767770 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:23.767777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:23.769481 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:23.769494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:23.770104 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:23.770111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:23.770382 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:23.771241 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:23.771266 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:23.773372 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:23.775466 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:23.775473 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:23.775491 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:23.775774 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:23.776110 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:23.776113 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:23.776115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:23.776383 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:23.776860 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:23.776868 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:23.777183 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 18:38:23.777444 (+ 261us) client_negotiation.cc:240] Negotiation successful
0807 18:38:23.777802 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":148,"threads_started":1}
I20260807 18:38:23.780146 12007 heartbeater.cc:347] Connected to a master server at 127.6.138.126:44397
I20260807 18:38:23.780437 12007 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:23.781152 12007 heartbeater.cc:511] Master 127.6.138.126:44397 requested a full tablet report, sending...
I20260807 18:38:23.782972 11698 ts_manager.cc:194] Registered new tserver with Master: 5dca27cc4da64ae8aa10afbeab0e2ea3 (127.6.138.65:41777)
I20260807 18:38:23.784617 11698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:42501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:23.797137 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44397
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:23.926244 12014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:23.926600 12014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:23.926702 12014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:23.931288 12014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:23.931360 12014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:23.931473 12014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:23.937757 12014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:23.939098 12014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:23.940320 12014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:23.950845 12022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:23.950896 12023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:23.950912 12026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:23.951128 12014 server_base.cc:1034] running on GCE node
I20260807 18:38:23.951764 12014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:23.952482 12014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:23.953658 12014 hybrid_clock.cc:648] HybridClock initialized: now 1786127903953642 us; error 27 us; skew 500 ppm
Aug 07 18:38:23 dist-test-slave-2d19 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:23.956684 12014 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:23.957631 12014 webserver.cc:460] Webserver started at http://127.6.138.66:36393/ using document root <none> and password file <none>
I20260807 18:38:23.958236 12014 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:23.958334 12014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:23.958577 12014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:23.960726 12014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "b60efd6e561c42bba216b084a32d9850"
format_stamp: "Formatted at 2026-08-07 18:38:23 on dist-test-slave-2d19"
server_key: "82273b411c66c8e3aa774fc5a7186058"
server_key_iv: "cba1616c5270a96ed909aba6badf7264"
server_key_version: "encryptionkey@0"
I20260807 18:38:23.961324 12014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "b60efd6e561c42bba216b084a32d9850"
format_stamp: "Formatted at 2026-08-07 18:38:23 on dist-test-slave-2d19"
server_key: "82273b411c66c8e3aa774fc5a7186058"
server_key_iv: "cba1616c5270a96ed909aba6badf7264"
server_key_version: "encryptionkey@0"
I20260807 18:38:23.965263 12014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:38:23.967679 12035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:23.968967 12014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:23.969144 12014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "b60efd6e561c42bba216b084a32d9850"
format_stamp: "Formatted at 2026-08-07 18:38:23 on dist-test-slave-2d19"
server_key: "82273b411c66c8e3aa774fc5a7186058"
server_key_iv: "cba1616c5270a96ed909aba6badf7264"
server_key_version: "encryptionkey@0"
I20260807 18:38:23.969264 12014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:23.991680 12014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:23.995193 12014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:23.995410 12014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:23.996058 12014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:23.996999 12014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:23.997082 12014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:23.997152 12014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:23.997200 12014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:24.007117 12014 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:39347
I20260807 18:38:24.007145 12217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:39347 every 8 connection(s)
I20260807 18:38:24.008031 12014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:24.016034 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 12014
I20260807 18:38:24.016135 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:24.016458 6697 external_mini_cluster.cc:1428] Setting key a80d116b364ce2c9805d65ef8d324a72
Aug 07 18:38:24 dist-test-slave-2d19 krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:24.023788 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:24.010015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:50491 (local address 127.6.138.126:44397)
0807 18:38:24.010191 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:24.010195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:24.011083 (+ 888us) server_negotiation.cc:407] Connection header received
0807 18:38:24.012657 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:24.012661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:24.012719 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:24.012833 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:24.014218 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.014798 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.015998 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.016383 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.016504 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:24.019491 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:24.019495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:24.019498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:24.019531 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:24.021407 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:24.021934 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:24.021941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:24.021946 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:24.022047 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:24.022398 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:24.022404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:24.022408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:24.022694 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:24.022822 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:24.023343 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 18:38:24.023515 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:38:24.024590 12223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:24.010434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44397 (local address 127.6.138.66:50491)
0807 18:38:24.010913 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:24.010959 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:24.012420 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:24.013026 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:24.013033 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:24.013539 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:24.014041 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:24.014056 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.014936 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.014942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:24.015863 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:24.015873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.016551 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.016555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:24.016744 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:24.017567 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:24.017588 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:24.019321 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:24.021550 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:24.021557 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:24.021570 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:24.021838 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:24.022148 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:24.022150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:24.022151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:24.022289 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:24.022808 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:24.022813 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:24.023073 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 18:38:24.023243 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 18:38:24.023510 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":124,"threads_started":1}
I20260807 18:38:24.025913 12219 heartbeater.cc:347] Connected to a master server at 127.6.138.126:44397
I20260807 18:38:24.026162 12219 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:24.026786 12219 heartbeater.cc:511] Master 127.6.138.126:44397 requested a full tablet report, sending...
I20260807 18:38:24.027910 11698 ts_manager.cc:194] Registered new tserver with Master: b60efd6e561c42bba216b084a32d9850 (127.6.138.66:39347)
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:38:24.028609 11698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:50491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:24.050071 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44397
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:24.172804 12226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:24.173064 12226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:24.173108 12226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:24.177155 12226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:24.177217 12226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:24.177309 12226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:24.182875 12226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:24.183993 12226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:24.185124 12226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:24.195343 12239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:24.195326 12242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:24.195479 12226 server_base.cc:1034] running on GCE node
W20260807 18:38:24.195326 12238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:24.196072 12226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:24.196805 12226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:24.198001 12226 hybrid_clock.cc:648] HybridClock initialized: now 1786127904197986 us; error 25 us; skew 500 ppm
Aug 07 18:38:24 dist-test-slave-2d19 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127904, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:24.201014 12226 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:24.201965 12226 webserver.cc:460] Webserver started at http://127.6.138.67:39673/ using document root <none> and password file <none>
I20260807 18:38:24.202533 12226 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:24.202596 12226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:24.202808 12226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:24.204926 12226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "c92e93c7e5bf44fe83e6b29c13b68b59"
format_stamp: "Formatted at 2026-08-07 18:38:24 on dist-test-slave-2d19"
server_key: "e03d674a972e6aa61f34f0f637584224"
server_key_iv: "d44cd71462ade073ec70f5cf6de72858"
server_key_version: "encryptionkey@0"
I20260807 18:38:24.205480 12226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "c92e93c7e5bf44fe83e6b29c13b68b59"
format_stamp: "Formatted at 2026-08-07 18:38:24 on dist-test-slave-2d19"
server_key: "e03d674a972e6aa61f34f0f637584224"
server_key_iv: "d44cd71462ade073ec70f5cf6de72858"
server_key_version: "encryptionkey@0"
I20260807 18:38:24.209529 12226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:38:24.211949 12253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:24.213187 12226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:24.213344 12226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "c92e93c7e5bf44fe83e6b29c13b68b59"
format_stamp: "Formatted at 2026-08-07 18:38:24 on dist-test-slave-2d19"
server_key: "e03d674a972e6aa61f34f0f637584224"
server_key_iv: "d44cd71462ade073ec70f5cf6de72858"
server_key_version: "encryptionkey@0"
I20260807 18:38:24.213459 12226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:24.224172 12226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:24.227526 12226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:24.227739 12226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:24.228436 12226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:24.229331 12226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:24.229379 12226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:24.229418 12226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:24.229441 12226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:24.238118 12226 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:46699
I20260807 18:38:24.238209 12436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:46699 every 8 connection(s)
I20260807 18:38:24.239087 12226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:24.247922 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 12226
I20260807 18:38:24.248090 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:24.248461 6697 external_mini_cluster.cc:1428] Setting key ca174d60bd04408c351edadc1d72680e
Aug 07 18:38:24 dist-test-slave-2d19 krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127904, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:24.254161 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:24.241050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:38063 (local address 127.6.138.126:44397)
0807 18:38:24.241196 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:24.241200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:24.242164 (+ 964us) server_negotiation.cc:407] Connection header received
0807 18:38:24.243614 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:24.243617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:24.243685 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:24.243782 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:24.245047 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.245586 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.246748 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.246988 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.247096 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:24.250522 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:24.250525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:24.250527 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:24.250555 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:24.252007 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:24.252609 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:24.252612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:24.252614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:24.252711 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:24.253042 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:24.253046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:24.253047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:24.253239 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:24.253331 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:24.253869 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 18:38:24.254040 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:38:24.255021 12441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:24.241440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44397 (local address 127.6.138.67:38063)
0807 18:38:24.241912 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:24.241956 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:24.243364 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:24.243962 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:24.243968 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:24.244415 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:24.244897 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:24.244910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.245727 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.245730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:24.246616 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:24.246625 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.247153 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.247158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:24.247324 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:24.248057 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:24.248118 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:24.250341 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:24.252144 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:24.252149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:24.252162 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:24.252503 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:24.252805 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:24.252807 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:24.252808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:24.252945 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:24.253334 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:24.253338 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:24.253602 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 18:38:24.253758 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 18:38:24.254025 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":118,"threads_started":1}
I20260807 18:38:24.256357 12438 heartbeater.cc:347] Connected to a master server at 127.6.138.126:44397
I20260807 18:38:24.256649 12438 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:24.257160 12438 heartbeater.cc:511] Master 127.6.138.126:44397 requested a full tablet report, sending...
I20260807 18:38:24.258210 11698 ts_manager.cc:194] Registered new tserver with Master: c92e93c7e5bf44fe83e6b29c13b68b59 (127.6.138.67:46699)
I20260807 18:38:24.258749 11698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:38063
I20260807 18:38:24.262782 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:38:24 dist-test-slave-2d19 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127904, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 18:38:24 dist-test-slave-2d19 krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127904, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:24.292438 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:24.283228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.6.138.126:44397)
0807 18:38:24.283363 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:24.283366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:24.283505 (+ 139us) server_negotiation.cc:407] Connection header received
0807 18:38:24.283627 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:24.283630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:24.283677 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:24.283763 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:24.284537 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.285139 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.286099 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:24.286351 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:24.286460 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:24.289095 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:24.289099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:24.289101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:24.289129 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:24.290585 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:24.291089 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:24.291092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:24.291093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:24.291173 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:24.291454 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:24.291457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:24.291458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:24.291688 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:24.291804 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:24.292105 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 18:38:24.292209 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 18:38:24.292925 11698 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42358
I20260807 18:38:24.294004 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 11793
I20260807 18:38:24.313161 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 12014
I20260807 18:38:24.330686 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 12226
I20260807 18:38:24.344378 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 11647
2026-08-07T18:38:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2542 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:24 dist-test-slave-2d19 krb5kdc[12456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:24 dist-test-slave-2d19 krb5kdc[12456](info): set up 2 sockets
Aug 07 18:38:24 dist-test-slave-2d19 krb5kdc[12456](info): commencing operation
krb5kdc: starting...
W20260807 18:38:25.612931 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:25 dist-test-slave-2d19 krb5kdc[12456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:25.751890 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37639
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:25.870275 12476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:25.870553 12476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:25.870622 12476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:25.874791 12476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:25.874858 12476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:25.874879 12476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:25.874938 12476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:25.874971 12476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:25.880303 12476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:37639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:25.881397 12476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:25.882462 12476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:25.891546 12489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:25.891575 12484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:25.891580 12485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:25.891747 12476 server_base.cc:1034] running on GCE node
I20260807 18:38:25.892587 12476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:25.893739 12476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:25.894909 12476 hybrid_clock.cc:648] HybridClock initialized: now 1786127905894898 us; error 30 us; skew 500 ppm
Aug 07 18:38:25 dist-test-slave-2d19 krb5kdc[12456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127905, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:25.897781 12476 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:25.898794 12476 webserver.cc:460] Webserver started at http://127.6.138.126:43457/ using document root <none> and password file <none>
I20260807 18:38:25.899391 12476 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:25.899488 12476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:25.899739 12476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:25.901907 12476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "3b0f119d0b62488c993284e57aad8d8e"
format_stamp: "Formatted at 2026-08-07 18:38:25 on dist-test-slave-2d19"
server_key: "c6f938b83fa18706b9e8087f632f9d24"
server_key_iv: "5a410c5162dc73d84ae214c45cd2d556"
server_key_version: "encryptionkey@0"
I20260807 18:38:25.902510 12476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "3b0f119d0b62488c993284e57aad8d8e"
format_stamp: "Formatted at 2026-08-07 18:38:25 on dist-test-slave-2d19"
server_key: "c6f938b83fa18706b9e8087f632f9d24"
server_key_iv: "5a410c5162dc73d84ae214c45cd2d556"
server_key_version: "encryptionkey@0"
I20260807 18:38:25.906378 12476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:25.908877 12504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:25.910187 12476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:25.910344 12476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "3b0f119d0b62488c993284e57aad8d8e"
format_stamp: "Formatted at 2026-08-07 18:38:25 on dist-test-slave-2d19"
server_key: "c6f938b83fa18706b9e8087f632f9d24"
server_key_iv: "5a410c5162dc73d84ae214c45cd2d556"
server_key_version: "encryptionkey@0"
I20260807 18:38:25.910460 12476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:25.924113 12476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:25.927109 12476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:25.927320 12476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:25.935257 12476 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:37639
I20260807 18:38:25.935288 12575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:37639 every 8 connection(s)
I20260807 18:38:25.936292 12476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:25.939152 12578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:25.939422 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 12476
I20260807 18:38:25.939561 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:25.939852 6697 external_mini_cluster.cc:1428] Setting key ecd31292158bad2c93c222554905b70e
I20260807 18:38:25.945818 12578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e: Bootstrap starting.
I20260807 18:38:25.948664 12578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e: Neither blocks nor log segments found. Creating new log.
Aug 07 18:38:25 dist-test-slave-2d19 krb5kdc[12456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:25.949630 12578 log.cc:824] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:25.952013 12578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e: No bootstrap required, opened a new log
I20260807 18:38:25.954514 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:25.941369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48836 (local address 127.6.138.126:37639)
0807 18:38:25.941816 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:25.941831 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:25.941867 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:38:25.942579 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:25.942597 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:25.942932 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:25.943291 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:25.944130 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:25.945176 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:25.946072 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:25.946727 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:25.946923 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:25.949773 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:25.949787 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:25.949802 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:25.949831 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:25.951543 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:25.952122 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:25.952131 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:25.952140 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:25.952307 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:25.952627 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:25.952630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:25.952631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:25.953015 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:25.953161 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:25.953471 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 18:38:25.953787 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":140,"threads_started":1}
I20260807 18:38:25.956212 12578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f119d0b62488c993284e57aad8d8e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37639 } }
I20260807 18:38:25.956512 12578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:25.956580 12578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b0f119d0b62488c993284e57aad8d8e, State: Initialized, Role: FOLLOWER
I20260807 18:38:25.957056 12578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f119d0b62488c993284e57aad8d8e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37639 } }
I20260807 18:38:25.957208 12578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:25.957250 12578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:25.957314 12578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:25.958155 12578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f119d0b62488c993284e57aad8d8e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37639 } }
I20260807 18:38:25.958454 12578 leader_election.cc:302] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b0f119d0b62488c993284e57aad8d8e; no voters:
I20260807 18:38:25.958730 12578 leader_election.cc:288] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:25.958884 12588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:25.959139 12588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [term 1 LEADER]: Becoming Leader. State: Replica: 3b0f119d0b62488c993284e57aad8d8e, State: Running, Role: LEADER
I20260807 18:38:25.959487 12588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f119d0b62488c993284e57aad8d8e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37639 } }
I20260807 18:38:25.959986 12578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:25.961278 12593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b0f119d0b62488c993284e57aad8d8e. Latest consensus state: current_term: 1 leader_uuid: "3b0f119d0b62488c993284e57aad8d8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f119d0b62488c993284e57aad8d8e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37639 } } }
I20260807 18:38:25.961410 12593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:25.961719 12591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b0f119d0b62488c993284e57aad8d8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0f119d0b62488c993284e57aad8d8e" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37639 } } }
I20260807 18:38:25.961819 12591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:25.961858 12605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:25.965011 12605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:25.971398 12605 catalog_manager.cc:1347] Generated new cluster ID: 2634de06b1dc499995a48d96d8d5fa15
I20260807 18:38:25.971474 12605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:25.990612 12605 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:25.991963 12605 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:26.000808 12605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b0f119d0b62488c993284e57aad8d8e: Generated new TSK 0
I20260807 18:38:26.001694 12605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:26.039601 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37639
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:26.156680 12621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:26.156940 12621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:26.156983 12621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:26.161092 12621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:26.161149 12621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:26.161242 12621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:26.166489 12621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:26.167570 12621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:26.168715 12621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:26.179476 12635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:26.179554 12621 server_base.cc:1034] running on GCE node
W20260807 18:38:26.179819 12632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:26.179524 12629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:26.180343 12621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:26.181011 12621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:26.182206 12621 hybrid_clock.cc:648] HybridClock initialized: now 1786127906182191 us; error 29 us; skew 500 ppm
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[12456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127906, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:26.185252 12621 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:26.186111 12621 webserver.cc:460] Webserver started at http://127.6.138.65:42905/ using document root <none> and password file <none>
I20260807 18:38:26.186743 12621 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:26.186810 12621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:26.186978 12621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:26.189113 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "cd7156c6a2b840df97b5f551f88e1327"
format_stamp: "Formatted at 2026-08-07 18:38:26 on dist-test-slave-2d19"
server_key: "57817016fd0f0194c9322500e0d546ec"
server_key_iv: "0b20763f7d84841cce893db871a8fcb1"
server_key_version: "encryptionkey@0"
I20260807 18:38:26.189649 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "cd7156c6a2b840df97b5f551f88e1327"
format_stamp: "Formatted at 2026-08-07 18:38:26 on dist-test-slave-2d19"
server_key: "57817016fd0f0194c9322500e0d546ec"
server_key_iv: "0b20763f7d84841cce893db871a8fcb1"
server_key_version: "encryptionkey@0"
I20260807 18:38:26.193450 12621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 18:38:26.195869 12647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:26.197208 12621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:26.197309 12621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "cd7156c6a2b840df97b5f551f88e1327"
format_stamp: "Formatted at 2026-08-07 18:38:26 on dist-test-slave-2d19"
server_key: "57817016fd0f0194c9322500e0d546ec"
server_key_iv: "0b20763f7d84841cce893db871a8fcb1"
server_key_version: "encryptionkey@0"
I20260807 18:38:26.197393 12621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:26.208469 12621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:26.211741 12621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:26.211901 12621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:26.212433 12621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:26.213218 12621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:26.213266 12621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:26.213303 12621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:26.213325 12621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:26.222247 12621 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:42389
I20260807 18:38:26.222314 12816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:42389 every 8 connection(s)
I20260807 18:38:26.223199 12621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:26.227062 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 12621
I20260807 18:38:26.227154 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:26.227463 6697 external_mini_cluster.cc:1428] Setting key 7dab5a3cd7252bbee3180f2acaff6cc6
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[12456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127906, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:26.239029 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:26.225149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:53355 (local address 127.6.138.126:37639)
0807 18:38:26.225284 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:26.225287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:26.226111 (+ 824us) server_negotiation.cc:407] Connection header received
0807 18:38:26.227665 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:26.227669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:26.227733 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:26.227842 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:26.229412 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.230022 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.231372 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.231622 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.231712 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:26.234654 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:26.234660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:26.234667 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:26.234704 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:26.236445 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.237273 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.237276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.237277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.237364 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.237791 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.237793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.237794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.237989 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:26.238069 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:26.238722 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 18:38:26.238851 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:38:26.240211 12819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:26.225506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37639 (local address 127.6.138.65:53355)
0807 18:38:26.225950 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:26.225994 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:26.227405 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:26.228298 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:26.228308 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:26.228687 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:26.229264 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:26.229278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.230160 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.230164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:26.231232 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:26.231247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.231790 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.231796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:26.232047 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:26.232730 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:26.232775 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:26.234476 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:26.236611 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:26.236616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:26.236661 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:26.237074 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:26.237476 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:26.237479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:26.237481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:26.237683 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:26.238108 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:26.238117 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:26.238448 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 18:38:26.238690 (+ 242us) client_negotiation.cc:240] Negotiation successful
0807 18:38:26.239134 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":109,"threads_started":1}
I20260807 18:38:26.241549 12817 heartbeater.cc:347] Connected to a master server at 127.6.138.126:37639
I20260807 18:38:26.241814 12817 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:26.242446 12817 heartbeater.cc:511] Master 127.6.138.126:37639 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
I20260807 18:38:26.244351 12533 ts_manager.cc:194] Registered new tserver with Master: cd7156c6a2b840df97b5f551f88e1327 (127.6.138.65:42389)
I20260807 18:38:26.245697 12533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:53355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:26.260095 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37639
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:26.378244 12825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:26.378553 12825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:26.378629 12825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:26.382699 12825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:26.382800 12825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:26.382893 12825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:26.388417 12825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:26.389597 12825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:26.390714 12825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:26.400857 12837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:26.400877 12834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:26.401221 12825 server_base.cc:1034] running on GCE node
W20260807 18:38:26.400862 12839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:26.401854 12825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:26.402523 12825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:26.403728 12825 hybrid_clock.cc:648] HybridClock initialized: now 1786127906403710 us; error 28 us; skew 500 ppm
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[12456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127906, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:26.406983 12825 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:26.407913 12825 webserver.cc:460] Webserver started at http://127.6.138.66:43237/ using document root <none> and password file <none>
I20260807 18:38:26.408686 12825 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:26.408783 12825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:26.409017 12825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:26.411090 12825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "2c26cbc982de48d9ac4fa4c739b31d61"
format_stamp: "Formatted at 2026-08-07 18:38:26 on dist-test-slave-2d19"
server_key: "9a878bb606ba30d04b57d356355841e7"
server_key_iv: "c99d4ce2235de9271f4af3ecbea478d4"
server_key_version: "encryptionkey@0"
I20260807 18:38:26.411684 12825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "2c26cbc982de48d9ac4fa4c739b31d61"
format_stamp: "Formatted at 2026-08-07 18:38:26 on dist-test-slave-2d19"
server_key: "9a878bb606ba30d04b57d356355841e7"
server_key_iv: "c99d4ce2235de9271f4af3ecbea478d4"
server_key_version: "encryptionkey@0"
I20260807 18:38:26.415665 12825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:38:26.418332 12850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:26.419822 12825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:26.420058 12825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "2c26cbc982de48d9ac4fa4c739b31d61"
format_stamp: "Formatted at 2026-08-07 18:38:26 on dist-test-slave-2d19"
server_key: "9a878bb606ba30d04b57d356355841e7"
server_key_iv: "c99d4ce2235de9271f4af3ecbea478d4"
server_key_version: "encryptionkey@0"
I20260807 18:38:26.420225 12825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:26.435714 12825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:26.439262 12825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:26.439487 12825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:26.440137 12825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:26.441123 12825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:26.441200 12825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:26.441272 12825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:26.441324 12825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:26.451427 12825 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:33871
I20260807 18:38:26.451452 13029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:33871 every 8 connection(s)
I20260807 18:38:26.452469 12825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:26.457901 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 12825
I20260807 18:38:26.458083 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:26.458393 6697 external_mini_cluster.cc:1428] Setting key b0ada19c2c901afa617df97c1f726bcd
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[12456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127906, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:26.467921 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:26.454518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:45597 (local address 127.6.138.126:37639)
0807 18:38:26.454657 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:26.454661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:26.455577 (+ 916us) server_negotiation.cc:407] Connection header received
0807 18:38:26.457053 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:26.457056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:26.457104 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:26.457187 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:26.458863 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.459486 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.460899 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.461134 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.461274 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:26.463874 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:26.463884 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:26.463887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:26.463909 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:26.465505 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.466178 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.466185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.466189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.466289 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.466642 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.466647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.466651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.466925 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:26.467058 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:26.467587 (+ 529us) server_negotiation.cc:299] Negotiation successful
0807 18:38:26.467724 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:38:26.468755 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:26.454924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37639 (local address 127.6.138.66:45597)
0807 18:38:26.455415 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:26.455463 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:26.456842 (+ 1379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:26.457388 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:26.457395 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:26.457922 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:26.458643 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:26.458666 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.459628 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.459632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:26.460766 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:26.460777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.461313 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.461317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:26.461514 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:26.462172 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:26.462189 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:26.463708 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:26.465655 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:26.465664 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:26.465683 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:26.466066 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:26.466394 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:26.466397 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:26.466398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:26.466539 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:26.467069 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:26.467075 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:26.467324 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 18:38:26.467497 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 18:38:26.467763 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:38:26.470188 13030 heartbeater.cc:347] Connected to a master server at 127.6.138.126:37639
I20260807 18:38:26.470453 13030 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:26.470973 13030 heartbeater.cc:511] Master 127.6.138.126:37639 requested a full tablet report, sending...
I20260807 18:38:26.472137 12533 ts_manager.cc:194] Registered new tserver with Master: 2c26cbc982de48d9ac4fa4c739b31d61 (127.6.138.66:33871)
I20260807 18:38:26.472810 12533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:45597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:26.491453 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37639
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:39945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:26.606679 13037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:26.606945 13037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:26.606990 13037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:26.611351 13037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:26.611420 13037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:26.611521 13037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:26.617076 13037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:39945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:26.618175 13037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:26.619231 13037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:26.629747 13051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:26.629741 13052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:26.629793 13037 server_base.cc:1034] running on GCE node
W20260807 18:38:26.629741 13055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:26.630481 13037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:26.631091 13037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:26.632328 13037 hybrid_clock.cc:648] HybridClock initialized: now 1786127906632309 us; error 28 us; skew 500 ppm
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[12456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127906, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:26.635295 13037 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:26.636194 13037 webserver.cc:460] Webserver started at http://127.6.138.67:36503/ using document root <none> and password file <none>
I20260807 18:38:26.636802 13037 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:26.636905 13037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:26.637157 13037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:26.639300 13037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "d7c43f12080c4820a03ba68486b2f823"
format_stamp: "Formatted at 2026-08-07 18:38:26 on dist-test-slave-2d19"
server_key: "8704a4c65a50ee031a31bc5193366c2f"
server_key_iv: "574beb0a6aff952890df16f056843d1e"
server_key_version: "encryptionkey@0"
I20260807 18:38:26.639884 13037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "d7c43f12080c4820a03ba68486b2f823"
format_stamp: "Formatted at 2026-08-07 18:38:26 on dist-test-slave-2d19"
server_key: "8704a4c65a50ee031a31bc5193366c2f"
server_key_iv: "574beb0a6aff952890df16f056843d1e"
server_key_version: "encryptionkey@0"
I20260807 18:38:26.643774 13037 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:38:26.646368 13064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:26.647670 13037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:26.647814 13037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "d7c43f12080c4820a03ba68486b2f823"
format_stamp: "Formatted at 2026-08-07 18:38:26 on dist-test-slave-2d19"
server_key: "8704a4c65a50ee031a31bc5193366c2f"
server_key_iv: "574beb0a6aff952890df16f056843d1e"
server_key_version: "encryptionkey@0"
I20260807 18:38:26.647933 13037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:26.681665 13037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:26.685245 13037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:26.685480 13037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:26.686146 13037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:26.687019 13037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:26.687103 13037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:26.687177 13037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:26.687222 13037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:26.696882 13037 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:36841
I20260807 18:38:26.696904 13244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:36841 every 8 connection(s)
I20260807 18:38:26.697837 13037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:26.699257 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 13037
I20260807 18:38:26.699398 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:26.699757 6697 external_mini_cluster.cc:1428] Setting key ad2e8eec707ac429301b967bb91c4605
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[12456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127906, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:26.711822 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:26.699790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:37115 (local address 127.6.138.126:37639)
0807 18:38:26.699949 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:26.699955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:26.701256 (+ 1301us) server_negotiation.cc:407] Connection header received
0807 18:38:26.702853 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:26.702855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:26.702901 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:26.702969 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:26.704136 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.704735 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.705865 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.706079 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.706159 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:26.708541 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:26.708544 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:26.708561 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:26.708586 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:26.710061 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.710506 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.710509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.710511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.710606 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.710898 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.710900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.710901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.711074 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:26.711150 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:26.711601 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 18:38:26.711700 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:38:26.712811 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:26.700349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37639 (local address 127.6.138.67:37115)
0807 18:38:26.701085 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:26.701144 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:26.702660 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:26.703157 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:26.703163 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:26.703508 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:26.704006 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:26.704020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.704846 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.704848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:26.705741 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:26.705751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.706212 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.706215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:26.706356 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:26.706892 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:26.706907 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:26.708389 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:26.710161 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:26.710166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:26.710177 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:26.710416 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:26.710688 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:26.710690 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:26.710691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:26.710817 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:26.711163 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:26.711168 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:26.711382 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 18:38:26.711537 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 18:38:26.711778 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":171,"threads_started":1}
I20260807 18:38:26.714064 13245 heartbeater.cc:347] Connected to a master server at 127.6.138.126:37639
I20260807 18:38:26.714322 13245 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:26.714828 13245 heartbeater.cc:511] Master 127.6.138.126:37639 requested a full tablet report, sending...
I20260807 18:38:26.715906 12533 ts_manager.cc:194] Registered new tserver with Master: d7c43f12080c4820a03ba68486b2f823 (127.6.138.67:36841)
I20260807 18:38:26.716569 12533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:37115
I20260807 18:38:26.726115 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[12456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[12456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:26.753576 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:26.743922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.6.138.65:42389)
0807 18:38:26.744222 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:26.744230 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:26.744291 (+ 61us) server_negotiation.cc:407] Connection header received
0807 18:38:26.744359 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:26.744365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:26.744571 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:26.744698 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:26.745433 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.746226 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.747061 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.747429 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.747513 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:26.749914 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:26.749924 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:26.749939 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:26.749965 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:26.751687 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.752198 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.752206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.752211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.752361 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.752612 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.752615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.752616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.752865 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:26.753046 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:26.753292 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 18:38:26.753426 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":123,"threads_started":1}
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[12456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:26.765204 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:26.756544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55052 (local address 127.6.138.126:37639)
0807 18:38:26.756702 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:26.756706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:26.756721 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:38:26.756835 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:26.756837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:26.756878 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:26.756945 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:26.757570 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.758084 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.758885 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:26.759107 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:26.759189 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:26.761794 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:26.761798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:26.761800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:26.761835 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:26.763416 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.763861 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.763864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.763866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.763954 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:26.764266 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:26.764273 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:26.764277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:26.764512 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:26.764666 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:26.764906 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 18:38:26.765034 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 18:38:26.765616 12533 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55052
I20260807 18:38:26.766237 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 12621
I20260807 18:38:26.789584 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 12825
I20260807 18:38:26.801467 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 13037
I20260807 18:38:26.816051 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 12476
2026-08-07T18:38:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2469 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[13267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[13267](info): set up 2 sockets
Aug 07 18:38:26 dist-test-slave-2d19 krb5kdc[13267](info): commencing operation
krb5kdc: starting...
W20260807 18:38:28.097034 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:28 dist-test-slave-2d19 krb5kdc[13267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:28.237520 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:36297
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:36297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:28.355073 13284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:28.355413 13284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:28.355507 13284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:28.359692 13284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:28.359792 13284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:28.359826 13284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:28.359869 13284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:28.359900 13284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:28.365546 13284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:36297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:36297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:28.366871 13284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:28.368057 13284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:28.377254 13294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:28.377259 13299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:28.377274 13290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:28.377635 13284 server_base.cc:1034] running on GCE node
I20260807 18:38:28.378255 13284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:28.379452 13284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:28.380659 13284 hybrid_clock.cc:648] HybridClock initialized: now 1786127908380632 us; error 48 us; skew 500 ppm
Aug 07 18:38:28 dist-test-slave-2d19 krb5kdc[13267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127908, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:28.389575 13284 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:28.390487 13284 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:28.390563 13284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:28.390761 13284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:28.393057 13284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "03c3edb6d3ce4f0bb302bc5337536c3a"
format_stamp: "Formatted at 2026-08-07 18:38:28 on dist-test-slave-2d19"
server_key: "9d4513e1b73d9d0d8d5eefa8acda08d0"
server_key_iv: "63442e99d731fbba6d0413ed71f0cc2e"
server_key_version: "encryptionkey@0"
I20260807 18:38:28.393606 13284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "03c3edb6d3ce4f0bb302bc5337536c3a"
format_stamp: "Formatted at 2026-08-07 18:38:28 on dist-test-slave-2d19"
server_key: "9d4513e1b73d9d0d8d5eefa8acda08d0"
server_key_iv: "63442e99d731fbba6d0413ed71f0cc2e"
server_key_version: "encryptionkey@0"
I20260807 18:38:28.397687 13284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:38:28.400354 13309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:28.401755 13284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:28.401897 13284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "03c3edb6d3ce4f0bb302bc5337536c3a"
format_stamp: "Formatted at 2026-08-07 18:38:28 on dist-test-slave-2d19"
server_key: "9d4513e1b73d9d0d8d5eefa8acda08d0"
server_key_iv: "63442e99d731fbba6d0413ed71f0cc2e"
server_key_version: "encryptionkey@0"
I20260807 18:38:28.402014 13284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:28.425937 13284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:28.429579 13284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:28.429814 13284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:28.437389 13284 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:36297
I20260807 18:38:28.437705 13387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:36297 every 8 connection(s)
I20260807 18:38:28.438118 13284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:28.441071 13388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:28.445531 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 13284
I20260807 18:38:28.445663 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:28.445966 6697 external_mini_cluster.cc:1428] Setting key b76f39cb9d17b727a774c58286f022fa
I20260807 18:38:28.447463 13388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a: Bootstrap starting.
I20260807 18:38:28.450042 13388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:28.450917 13388 log.cc:824] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:28.453276 13388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a: No bootstrap required, opened a new log
Aug 07 18:38:28 dist-test-slave-2d19 krb5kdc[13267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:28.457655 13388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3edb6d3ce4f0bb302bc5337536c3a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36297 } }
I20260807 18:38:28.457950 13388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:28.458014 13388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c3edb6d3ce4f0bb302bc5337536c3a, State: Initialized, Role: FOLLOWER
I20260807 18:38:28.458571 13388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3edb6d3ce4f0bb302bc5337536c3a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36297 } }
I20260807 18:38:28.458757 13388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:28.458832 13388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:28.458935 13388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:28.460140 13388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3edb6d3ce4f0bb302bc5337536c3a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36297 } }
I20260807 18:38:28.460522 13388 leader_election.cc:302] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03c3edb6d3ce4f0bb302bc5337536c3a; no voters:
I20260807 18:38:28.460868 13388 leader_election.cc:288] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:28.460976 13398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:28.461231 13398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [term 1 LEADER]: Becoming Leader. State: Replica: 03c3edb6d3ce4f0bb302bc5337536c3a, State: Running, Role: LEADER
I20260807 18:38:28.461357 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:28.447461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.6.138.126:36297)
0807 18:38:28.447965 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:28.447993 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:28.448032 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:38:28.448780 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:28.448800 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:28.449162 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:28.449456 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:28.450397 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.451442 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.452439 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.453125 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.453313 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:28.455795 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:28.455802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:28.455821 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:28.455860 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:28.458045 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:28.458478 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:28.458485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:28.458490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:28.458638 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:28.458987 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:28.458992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:28.458993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:28.459530 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:28.459731 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:28.459967 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 18:38:28.460333 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":162,"threads_started":1}
I20260807 18:38:28.461592 13398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3edb6d3ce4f0bb302bc5337536c3a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36297 } }
I20260807 18:38:28.462231 13388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:28.463184 13401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03c3edb6d3ce4f0bb302bc5337536c3a. Latest consensus state: current_term: 1 leader_uuid: "03c3edb6d3ce4f0bb302bc5337536c3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3edb6d3ce4f0bb302bc5337536c3a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36297 } } }
I20260807 18:38:28.463317 13401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:28.463653 13399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03c3edb6d3ce4f0bb302bc5337536c3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c3edb6d3ce4f0bb302bc5337536c3a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36297 } } }
I20260807 18:38:28.463729 13399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:28.464143 13415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:28.466848 13415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:28.472061 13415 catalog_manager.cc:1347] Generated new cluster ID: a74591d7d3ca428c8f719e932392f38c
I20260807 18:38:28.472131 13415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:28.479959 13415 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:28.481263 13415 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:28.490975 13415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a: Generated new TSK 0
I20260807 18:38:28.491776 13415 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:28.531510 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:36297
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:28.647926 13433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:28.648195 13433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:28.648242 13433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:28.652359 13433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:28.652428 13433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:28.652491 13433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:28.658166 13433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:36297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:28.659302 13433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:28.660331 13433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:28.670377 13441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:28.670367 13442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:28.670390 13447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:28.670369 13433 server_base.cc:1034] running on GCE node
I20260807 18:38:28.670967 13433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:28.671675 13433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:28.672870 13433 hybrid_clock.cc:648] HybridClock initialized: now 1786127908672858 us; error 35 us; skew 500 ppm
Aug 07 18:38:28 dist-test-slave-2d19 krb5kdc[13267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127908, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:28.675686 13433 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:28.676510 13433 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:28.676609 13433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:28.676856 13433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:28.678967 13433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "3575020d19b0490db85819b9b477e783"
format_stamp: "Formatted at 2026-08-07 18:38:28 on dist-test-slave-2d19"
server_key: "f9e90fd5b057aa2d7187fc3dbe3763e7"
server_key_iv: "c3cab08d7c2603e14a55b490c750c15d"
server_key_version: "encryptionkey@0"
I20260807 18:38:28.679569 13433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "3575020d19b0490db85819b9b477e783"
format_stamp: "Formatted at 2026-08-07 18:38:28 on dist-test-slave-2d19"
server_key: "f9e90fd5b057aa2d7187fc3dbe3763e7"
server_key_iv: "c3cab08d7c2603e14a55b490c750c15d"
server_key_version: "encryptionkey@0"
I20260807 18:38:28.683341 13433 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 18:38:28.685876 13456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:28.687065 13433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:28.687206 13433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "3575020d19b0490db85819b9b477e783"
format_stamp: "Formatted at 2026-08-07 18:38:28 on dist-test-slave-2d19"
server_key: "f9e90fd5b057aa2d7187fc3dbe3763e7"
server_key_iv: "c3cab08d7c2603e14a55b490c750c15d"
server_key_version: "encryptionkey@0"
I20260807 18:38:28.687327 13433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:28.710333 13433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:28.713922 13433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:28.714159 13433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:28.714815 13433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:28.715710 13433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:28.715785 13433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:28.715858 13433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:28.715903 13433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:28.725178 13433 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:35691
I20260807 18:38:28.725198 13634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:35691 every 8 connection(s)
I20260807 18:38:28.725839 13433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:28.728911 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 13433
I20260807 18:38:28.729055 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:28.729414 6697 external_mini_cluster.cc:1428] Setting key d3c325ff9a7d80075badd617941d49cd
Aug 07 18:38:28 dist-test-slave-2d19 krb5kdc[13267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127908, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:38:28.741245 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:28.727686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:54765 (local address 127.6.138.126:36297)
0807 18:38:28.727826 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:28.727830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:28.729280 (+ 1450us) server_negotiation.cc:407] Connection header received
0807 18:38:28.730278 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:28.730281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:28.730327 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:28.730417 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:28.731771 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.732374 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.733817 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.734234 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.734362 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:28.737275 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:28.737278 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:28.737280 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:28.737305 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:28.738872 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:28.739545 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:28.739548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:28.739549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:28.739625 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:28.740060 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:28.740062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:28.740063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:28.740235 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:28.740378 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:28.740995 (+ 617us) server_negotiation.cc:299] Negotiation successful
0807 18:38:28.741122 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:38:28.742215 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:28.728066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:36297 (local address 127.6.138.65:54765)
0807 18:38:28.729110 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:28.729171 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:28.730099 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:28.730549 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:28.730555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:28.730922 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:28.731609 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:28.731633 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.732503 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.732510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:28.733686 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:28.733696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.734385 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.734387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:28.734542 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:28.735176 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:28.735192 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:28.737092 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:28.739003 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:28.739012 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:28.739031 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:28.739427 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:28.739728 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:28.739733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:28.739738 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:28.739939 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:28.740439 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:28.740448 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:28.740771 (+ 323us) client_negotiation.cc:769] Sending connection context
0807 18:38:28.740946 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 18:38:28.741223 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":176,"threads_started":1}
I20260807 18:38:28.743566 13635 heartbeater.cc:347] Connected to a master server at 127.6.138.126:36297
I20260807 18:38:28.743845 13635 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
I20260807 18:38:28.744424 13635 heartbeater.cc:511] Master 127.6.138.126:36297 requested a full tablet report, sending...
I20260807 18:38:28.746205 13339 ts_manager.cc:194] Registered new tserver with Master: 3575020d19b0490db85819b9b477e783 (127.6.138.65:35691)
I20260807 18:38:28.747394 13339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:54765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:28.763469 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:36297
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:28.881212 13645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:28.881476 13645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:28.881544 13645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:28.885543 13645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:28.885600 13645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:28.885663 13645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:28.891121 13645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:36297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:28.892218 13645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:28.893339 13645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:28.903054 13651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:28.903098 13652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:28.903017 13657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:28.903288 13645 server_base.cc:1034] running on GCE node
I20260807 18:38:28.903708 13645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:28.904441 13645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:28.905627 13645 hybrid_clock.cc:648] HybridClock initialized: now 1786127908905595 us; error 41 us; skew 500 ppm
Aug 07 18:38:28 dist-test-slave-2d19 krb5kdc[13267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127908, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:28.908553 13645 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:28.909240 13645 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:28.909301 13645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:28.909480 13645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:28.911450 13645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "a128dfadfd4747bcabe956f39dc82b68"
format_stamp: "Formatted at 2026-08-07 18:38:28 on dist-test-slave-2d19"
server_key: "1b26b0f75ea93f420be4cb8b65b6657a"
server_key_iv: "a6069edee2c1ff34d9017886a9db016b"
server_key_version: "encryptionkey@0"
I20260807 18:38:28.911972 13645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "a128dfadfd4747bcabe956f39dc82b68"
format_stamp: "Formatted at 2026-08-07 18:38:28 on dist-test-slave-2d19"
server_key: "1b26b0f75ea93f420be4cb8b65b6657a"
server_key_iv: "a6069edee2c1ff34d9017886a9db016b"
server_key_version: "encryptionkey@0"
I20260807 18:38:28.915628 13645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 18:38:28.918077 13666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:28.919293 13645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:38:28.919391 13645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "a128dfadfd4747bcabe956f39dc82b68"
format_stamp: "Formatted at 2026-08-07 18:38:28 on dist-test-slave-2d19"
server_key: "1b26b0f75ea93f420be4cb8b65b6657a"
server_key_iv: "a6069edee2c1ff34d9017886a9db016b"
server_key_version: "encryptionkey@0"
I20260807 18:38:28.919497 13645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:28.938112 13645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:28.941301 13645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:28.941468 13645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:28.942015 13645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:28.942826 13645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:28.942873 13645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:28.942921 13645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:28.942938 13645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:28.952297 13645 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:41877
I20260807 18:38:28.952329 13854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:41877 every 8 connection(s)
I20260807 18:38:28.952970 13645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:28.960767 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 13645
I20260807 18:38:28.960906 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:28.961189 6697 external_mini_cluster.cc:1428] Setting key 310c9add7483156821cee1a14f9c4f50
Aug 07 18:38:28 dist-test-slave-2d19 krb5kdc[13267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127908, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:28.968223 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:28.954894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:40625 (local address 127.6.138.126:36297)
0807 18:38:28.955045 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:28.955050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:28.955961 (+ 911us) server_negotiation.cc:407] Connection header received
0807 18:38:28.957317 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:28.957319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:28.957367 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:28.957445 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:28.958632 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.959162 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.960303 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.960683 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.960806 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:28.963791 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:28.963798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:28.963804 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:28.963841 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:28.965686 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:28.966391 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:28.966394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:28.966395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:28.966477 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:28.966893 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:28.966899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:28.966903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:28.967177 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:28.967288 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:28.967795 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 18:38:28.967944 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:38:28.968990 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:28.955312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:36297 (local address 127.6.138.66:40625)
0807 18:38:28.955797 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:28.955843 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:28.957110 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:28.957628 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:28.957635 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:28.958003 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:28.958480 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:28.958494 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.959275 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.959278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:28.960160 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:28.960170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:28.960852 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:28.960856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:28.961094 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:28.961805 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:28.961829 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:28.963551 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:28.965820 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:28.965830 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:28.965851 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:28.966280 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:28.966575 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:28.966579 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:28.966584 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:28.966793 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:28.967288 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:28.967293 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:28.967539 (+ 246us) client_negotiation.cc:769] Sending connection context
0807 18:38:28.967703 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 18:38:28.967972 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":124,"threads_started":1}
I20260807 18:38:28.970336 13855 heartbeater.cc:347] Connected to a master server at 127.6.138.126:36297
I20260807 18:38:28.970631 13855 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:28.971232 13855 heartbeater.cc:511] Master 127.6.138.126:36297 requested a full tablet report, sending...
I20260807 18:38:28.972522 13339 ts_manager.cc:194] Registered new tserver with Master: a128dfadfd4747bcabe956f39dc82b68 (127.6.138.66:41877)
I20260807 18:38:28.973111 13339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:40625
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:28.996541 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:36297
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:29.121016 13860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:29.121280 13860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:29.121326 13860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:29.125525 13860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:29.125593 13860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:29.125665 13860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:29.131639 13860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:36297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:29.132912 13860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:29.134016 13860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:29.143869 13868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:29.143890 13872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:29.143864 13867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:29.144133 13860 server_base.cc:1034] running on GCE node
I20260807 18:38:29.144603 13860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:29.145267 13860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:29.146438 13860 hybrid_clock.cc:648] HybridClock initialized: now 1786127909146411 us; error 38 us; skew 500 ppm
Aug 07 18:38:29 dist-test-slave-2d19 krb5kdc[13267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127909, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:29.149461 13860 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:29.150236 13860 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:29.150341 13860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:29.150594 13860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:29.152796 13860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "c4100644d4c74d5491fed65eb12df9bf"
format_stamp: "Formatted at 2026-08-07 18:38:29 on dist-test-slave-2d19"
server_key: "72ef6202d1af68c0e684196845258346"
server_key_iv: "1d3f3abb2d2dc00c6a41e431c1337ed0"
server_key_version: "encryptionkey@0"
I20260807 18:38:29.153378 13860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "c4100644d4c74d5491fed65eb12df9bf"
format_stamp: "Formatted at 2026-08-07 18:38:29 on dist-test-slave-2d19"
server_key: "72ef6202d1af68c0e684196845258346"
server_key_iv: "1d3f3abb2d2dc00c6a41e431c1337ed0"
server_key_version: "encryptionkey@0"
I20260807 18:38:29.157531 13860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 18:38:29.160106 13878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:29.161507 13860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:29.161660 13860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "c4100644d4c74d5491fed65eb12df9bf"
format_stamp: "Formatted at 2026-08-07 18:38:29 on dist-test-slave-2d19"
server_key: "72ef6202d1af68c0e684196845258346"
server_key_iv: "1d3f3abb2d2dc00c6a41e431c1337ed0"
server_key_version: "encryptionkey@0"
I20260807 18:38:29.161782 13860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:29.187489 13860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:29.191692 13860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:29.191937 13860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:29.192691 13860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:29.193632 13860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:29.193686 13860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:29.193768 13860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:29.193814 13860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:29.203253 13860 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:40865
I20260807 18:38:29.203286 14080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:40865 every 8 connection(s)
I20260807 18:38:29.203909 13860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:29.205060 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 13860
I20260807 18:38:29.205190 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:29.205525 6697 external_mini_cluster.cc:1428] Setting key 58c54828fb8542eaccae33426f0fa96c
Aug 07 18:38:29 dist-test-slave-2d19 krb5kdc[13267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127909, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:29.218488 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.206063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:38769 (local address 127.6.138.126:36297)
0807 18:38:29.206213 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.206218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.207881 (+ 1663us) server_negotiation.cc:407] Connection header received
0807 18:38:29.208864 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.208867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.208916 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.208996 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:29.210151 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.210740 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.211888 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.212116 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.212209 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.214905 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:29.214908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:29.214911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:29.214936 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:29.216444 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:29.216953 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:29.216957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:29.216958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:29.217045 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:29.217351 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:29.217353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:29.217354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:29.217561 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:29.217646 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.218144 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.218280 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:38:29.219461 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.206713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:36297 (local address 127.6.138.67:38769)
0807 18:38:29.207710 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:29.207777 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:29.208654 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:29.209135 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:29.209141 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:29.209496 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:29.210016 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.210030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.210853 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.210856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.211733 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.211742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.212308 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.212313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.212460 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.213069 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:29.213086 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:29.214718 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:29.216588 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:29.216593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:29.216605 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:29.216861 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:29.217129 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:29.217131 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:29.217132 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:29.217265 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:29.217655 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:29.217660 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:29.217885 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 18:38:29.218096 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 18:38:29.218406 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":203,"threads_started":1}
I20260807 18:38:29.220616 14081 heartbeater.cc:347] Connected to a master server at 127.6.138.126:36297
I20260807 18:38:29.220871 14081 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:29.221341 14081 heartbeater.cc:511] Master 127.6.138.126:36297 requested a full tablet report, sending...
I20260807 18:38:29.222504 13339 ts_manager.cc:194] Registered new tserver with Master: c4100644d4c74d5491fed65eb12df9bf (127.6.138.67:40865)
I20260807 18:38:29.223112 13339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:38769
I20260807 18:38:29.232210 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:38:29.243842 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.234833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56070 (local address 127.6.138.126:36297)
0807 18:38:29.235162 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.235167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.235266 (+ 99us) server_negotiation.cc:407] Connection header received
0807 18:38:29.235441 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.235444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.235515 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.235605 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:29.236582 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.237422 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.238730 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.239115 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.239231 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.240786 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:29.240789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:29.240791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:29.240816 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:29.242174 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:29.242572 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:29.242575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:29.242576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:29.242656 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:29.242954 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:29.242956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:29.242957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:29.243179 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:29.243291 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.243498 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.243630 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260807 18:38:29.246754 13339 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:29.249929 13339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:29.266088 14106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.259128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:40865 (local address 127.0.0.1:46054)
0807 18:38:29.260405 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:29.260459 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:29.260600 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:29.261198 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:29.261204 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:29.261230 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:29.261366 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.261373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.262821 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.262825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.264575 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.264590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.265462 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.265466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.265596 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.265642 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:38:29.265784 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 18:38:29.265958 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":133,"thread_start_us":84,"threads_started":1}
I20260807 18:38:29.266093 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.259295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.6.138.67:40865)
0807 18:38:29.260510 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.260527 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.260557 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:38:29.260784 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.260790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.261010 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.261178 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:29.261508 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.262667 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.264732 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.265313 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.265450 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.265608 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.265818 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.265921 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1093,"thread_start_us":189,"threads_started":1}
I20260807 18:38:29.266660 14105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.259550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:41877 (local address 127.0.0.1:39854)
0807 18:38:29.260673 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:29.260691 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:29.260772 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:29.261753 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:29.261756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:29.261776 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:29.261872 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.261879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.263400 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.263403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.264663 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.264679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.266222 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.266226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.266315 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.266338 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:38:29.266413 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 18:38:29.266494 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"mutex_wait_us":68,"spinlock_wait_cycles":17152,"thread_start_us":788,"threads_started":1}
I20260807 18:38:29.266898 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.259236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:35691 (local address 127.0.0.1:33720)
0807 18:38:29.260642 (+ 1406us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:29.260656 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:29.260772 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:29.261496 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:29.261499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:29.261518 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:29.261617 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.261623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.263069 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.263075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.264617 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.264627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.266379 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.266383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.266470 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.266500 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:38:29.266580 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:38:29.266655 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1338,"mutex_wait_us":161,"spinlock_wait_cycles":1280,"thread_start_us":103,"threads_started":1}
I20260807 18:38:29.267076 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.259958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.6.138.65:35691)
0807 18:38:29.260919 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.260926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.260948 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:38:29.261020 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.261026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.261201 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.261332 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:29.261758 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.262855 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.264761 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.265669 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.265796 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.265950 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.266674 (+ 724us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.266919 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":100,"threads_started":1}
I20260807 18:38:29.267382 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.259981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.6.138.66:41877)
0807 18:38:29.261061 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.261071 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.261090 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:38:29.261197 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.261203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.261409 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.261574 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:29.262028 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.263207 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.264782 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.265665 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.266132 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.266982 (+ 850us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.267035 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.267143 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":978,"thread_start_us":171,"threads_started":1}
I20260807 18:38:29.269937 13969 tablet_service.cc:1467] Processing CreateTablet for tablet a78afef7b421437cb7b3fe641f10e904 (DEFAULT_TABLE table=test-table [id=4daa6d22c536472bb3029e48731e53e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:29.270124 13530 tablet_service.cc:1467] Processing CreateTablet for tablet a78afef7b421437cb7b3fe641f10e904 (DEFAULT_TABLE table=test-table [id=4daa6d22c536472bb3029e48731e53e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:29.270733 13746 tablet_service.cc:1467] Processing CreateTablet for tablet a78afef7b421437cb7b3fe641f10e904 (DEFAULT_TABLE table=test-table [id=4daa6d22c536472bb3029e48731e53e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:29.271170 13969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a78afef7b421437cb7b3fe641f10e904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:29.271204 13530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a78afef7b421437cb7b3fe641f10e904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:29.271699 13746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a78afef7b421437cb7b3fe641f10e904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:29.278596 14116 tablet_bootstrap.cc:492] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: Bootstrap starting.
I20260807 18:38:29.279541 14120 tablet_bootstrap.cc:492] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: Bootstrap starting.
I20260807 18:38:29.280624 14119 tablet_bootstrap.cc:492] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783: Bootstrap starting.
I20260807 18:38:29.281774 14116 tablet_bootstrap.cc:654] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:29.281801 14120 tablet_bootstrap.cc:654] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:29.282749 14119 tablet_bootstrap.cc:654] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:29.282991 14116 log.cc:824] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:29.283079 14120 log.cc:824] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:29.283541 14119 log.cc:824] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:29.285192 14116 tablet_bootstrap.cc:492] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: No bootstrap required, opened a new log
I20260807 18:38:29.285369 14119 tablet_bootstrap.cc:492] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783: No bootstrap required, opened a new log
I20260807 18:38:29.285419 14116 ts_tablet_manager.cc:1397] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260807 18:38:29.285604 14119 ts_tablet_manager.cc:1397] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:38:29.289148 14116 raft_consensus.cc:348] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } }
I20260807 18:38:29.289434 14116 raft_consensus.cc:374] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:29.289506 14116 raft_consensus.cc:729] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4100644d4c74d5491fed65eb12df9bf, State: Initialized, Role: FOLLOWER
I20260807 18:38:29.289570 14119 raft_consensus.cc:348] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } }
I20260807 18:38:29.289757 14120 tablet_bootstrap.cc:492] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: No bootstrap required, opened a new log
I20260807 18:38:29.289819 14119 raft_consensus.cc:374] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:29.289872 14119 raft_consensus.cc:729] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3575020d19b0490db85819b9b477e783, State: Initialized, Role: FOLLOWER
I20260807 18:38:29.289953 14120 ts_tablet_manager.cc:1397] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.004s
I20260807 18:38:29.290042 14116 consensus_queue.cc:260] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } }
I20260807 18:38:29.290299 14119 consensus_queue.cc:260] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } }
I20260807 18:38:29.291078 13635 heartbeater.cc:503] Master 127.6.138.126:36297 was elected leader, sending a full tablet report...
I20260807 18:38:29.291332 14119 ts_tablet_manager.cc:1428] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260807 18:38:29.291707 14116 ts_tablet_manager.cc:1428] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:38:29.292107 14081 heartbeater.cc:503] Master 127.6.138.126:36297 was elected leader, sending a full tablet report...
I20260807 18:38:29.293885 14120 raft_consensus.cc:348] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } }
I20260807 18:38:29.294200 14120 raft_consensus.cc:374] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:29.294272 14120 raft_consensus.cc:729] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a128dfadfd4747bcabe956f39dc82b68, State: Initialized, Role: FOLLOWER
I20260807 18:38:29.294734 14120 consensus_queue.cc:260] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } }
I20260807 18:38:29.295431 13855 heartbeater.cc:503] Master 127.6.138.126:36297 was elected leader, sending a full tablet report...
I20260807 18:38:29.295753 14120 ts_tablet_manager.cc:1428] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 18:38:29.349402 14126 raft_consensus.cc:482] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:29.349696 14126 raft_consensus.cc:504] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } }
I20260807 18:38:29.351080 14126 leader_election.cc:288] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3575020d19b0490db85819b9b477e783 (127.6.138.65:35691), a128dfadfd4747bcabe956f39dc82b68 (127.6.138.66:41877)
I20260807 18:38:29.355212 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.351297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:35691 (local address 127.6.138.67:50881)
0807 18:38:29.351565 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:29.351586 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:29.351701 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:29.352038 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:29.352044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:29.352102 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:29.352240 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.352250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.353173 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.353177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.354030 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.354040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.354745 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.354749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.354864 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.354888 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:38:29.354962 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:38:29.355040 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":24}
I20260807 18:38:29.355212 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.351453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:50881 (local address 127.6.138.65:35691)
0807 18:38:29.351589 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.351596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.351753 (+ 157us) server_negotiation.cc:407] Connection header received
0807 18:38:29.351827 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.351831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.351912 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.352009 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:29.352392 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.353035 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.354182 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.354611 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.354750 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.354809 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.354990 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.355086 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:38:29.355345 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.351570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:51239 (local address 127.6.138.66:41877)
0807 18:38:29.351703 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.351710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.352020 (+ 310us) server_negotiation.cc:407] Connection header received
0807 18:38:29.352134 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.352137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.352182 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.352299 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:29.352600 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.353072 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.354219 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.354745 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.354865 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.354913 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.355145 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.355225 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:38:29.355752 14132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.351427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:41877 (local address 127.6.138.67:51239)
0807 18:38:29.351902 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:29.351916 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:29.352027 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:29.352328 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:29.352331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:29.352352 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:29.352475 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.352482 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.353221 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.353223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.354056 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:29.354066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.354903 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.354906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:29.355008 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.355027 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:38:29.355538 (+ 511us) client_negotiation.cc:240] Negotiation successful
0807 18:38:29.355613 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":23,"thread_start_us":165,"threads_started":1}
I20260807 18:38:29.355950 13567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78afef7b421437cb7b3fe641f10e904" candidate_uuid: "c4100644d4c74d5491fed65eb12df9bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3575020d19b0490db85819b9b477e783" is_pre_election: true
I20260807 18:38:29.356406 13567 raft_consensus.cc:2454] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4100644d4c74d5491fed65eb12df9bf in term 0.
I20260807 18:38:29.356460 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78afef7b421437cb7b3fe641f10e904" candidate_uuid: "c4100644d4c74d5491fed65eb12df9bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a128dfadfd4747bcabe956f39dc82b68" is_pre_election: true
I20260807 18:38:29.356752 13779 raft_consensus.cc:2454] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4100644d4c74d5491fed65eb12df9bf in term 0.
I20260807 18:38:29.356982 13879 leader_election.cc:302] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3575020d19b0490db85819b9b477e783, c4100644d4c74d5491fed65eb12df9bf; no voters:
I20260807 18:38:29.357292 14126 raft_consensus.cc:2781] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:29.357384 14126 raft_consensus.cc:482] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:29.357434 14126 raft_consensus.cc:3037] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:29.358549 14126 raft_consensus.cc:504] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } }
I20260807 18:38:29.358929 14126 leader_election.cc:288] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [CANDIDATE]: Term 1 election: Requested vote from peers 3575020d19b0490db85819b9b477e783 (127.6.138.65:35691), a128dfadfd4747bcabe956f39dc82b68 (127.6.138.66:41877)
I20260807 18:38:29.359298 13567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78afef7b421437cb7b3fe641f10e904" candidate_uuid: "c4100644d4c74d5491fed65eb12df9bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3575020d19b0490db85819b9b477e783"
I20260807 18:38:29.359361 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a78afef7b421437cb7b3fe641f10e904" candidate_uuid: "c4100644d4c74d5491fed65eb12df9bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a128dfadfd4747bcabe956f39dc82b68"
I20260807 18:38:29.359443 13567 raft_consensus.cc:3037] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:29.359475 13779 raft_consensus.cc:3037] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:29.360883 13779 raft_consensus.cc:2454] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4100644d4c74d5491fed65eb12df9bf in term 1.
I20260807 18:38:29.360883 13567 raft_consensus.cc:2454] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4100644d4c74d5491fed65eb12df9bf in term 1.
I20260807 18:38:29.361250 13879 leader_election.cc:302] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3575020d19b0490db85819b9b477e783, c4100644d4c74d5491fed65eb12df9bf; no voters:
I20260807 18:38:29.361497 14126 raft_consensus.cc:2781] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:29.361775 14126 raft_consensus.cc:686] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 1 LEADER]: Becoming Leader. State: Replica: c4100644d4c74d5491fed65eb12df9bf, State: Running, Role: LEADER
I20260807 18:38:29.362170 14126 consensus_queue.cc:237] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } }
I20260807 18:38:29.365911 13336 catalog_manager.cc:5495] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf reported cstate change: term changed from 0 to 1, leader changed from <none> to c4100644d4c74d5491fed65eb12df9bf (127.6.138.67). New cstate: current_term: 1 leader_uuid: "c4100644d4c74d5491fed65eb12df9bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4100644d4c74d5491fed65eb12df9bf" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 } health_report { overall_health: UNKNOWN } } }
I20260807 18:38:29.388695 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.384860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.6.138.67:40865)
0807 18:38:29.385051 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.385055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.385071 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:29.385156 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.385158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.385217 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.385317 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:29.385568 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.386182 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.387031 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:29.387269 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:29.387362 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:29.387649 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:29.388140 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:29.388246 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.388394 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.388456 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:38:29.395984 13567 raft_consensus.cc:1261] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783 [term 1 FOLLOWER]: Refusing update from remote peer c4100644d4c74d5491fed65eb12df9bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:29.395953 13779 raft_consensus.cc:1261] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [term 1 FOLLOWER]: Refusing update from remote peer c4100644d4c74d5491fed65eb12df9bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:29.396916 14134 consensus_queue.cc:1035] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a128dfadfd4747bcabe956f39dc82b68" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 41877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:29.397208 14126 consensus_queue.cc:1035] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3575020d19b0490db85819b9b477e783" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:29.405643 14138 mvcc.cc:205] Tried to move back new op lower bound from 7315979916876976128 to 7315979916751167488. Current Snapshot: MvccSnapshot[applied={T|T < 7315979916876976128}]
I20260807 18:38:29.417661 13335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:29.417891 13335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:29.420686 13335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03c3edb6d3ce4f0bb302bc5337536c3a: Sending DeleteTablet for 3 replicas of tablet a78afef7b421437cb7b3fe641f10e904
I20260807 18:38:29.421612 13969 tablet_service.cc:1514] Processing DeleteTablet for tablet a78afef7b421437cb7b3fe641f10e904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:29 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:46054
I20260807 18:38:29.421602 13530 tablet_service.cc:1514] Processing DeleteTablet for tablet a78afef7b421437cb7b3fe641f10e904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:29 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:33720
I20260807 18:38:29.421767 13746 tablet_service.cc:1514] Processing DeleteTablet for tablet a78afef7b421437cb7b3fe641f10e904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:29 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:39854
I20260807 18:38:29.422294 14150 tablet_replica.cc:330] T a78afef7b421437cb7b3fe641f10e904 P 3575020d19b0490db85819b9b477e783: stopping tablet replica
I20260807 18:38:29.422462 14152 tablet_replica.cc:330] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: stopping tablet replica
I20260807 18:38:29.422840 14152 raft_consensus.cc:2229] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:29.423035 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 13433
I20260807 18:38:29.423316 14152 raft_consensus.cc:2258] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:29.423774 14153 tablet_replica.cc:330] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: stopping tablet replica
I20260807 18:38:29.424093 14153 raft_consensus.cc:2229] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:29.424458 14153 raft_consensus.cc:2258] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:29.427742 14153 ts_tablet_manager.cc:1905] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:29.429495 14152 ts_tablet_manager.cc:1905] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:29.431036 14153 ts_tablet_manager.cc:1918] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:29.431236 14153 log.cc:1196] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/a78afef7b421437cb7b3fe641f10e904
I20260807 18:38:29.431706 14153 ts_tablet_manager.cc:1939] T a78afef7b421437cb7b3fe641f10e904 P a128dfadfd4747bcabe956f39dc82b68: Deleting consensus metadata
I20260807 18:38:29.432701 14152 ts_tablet_manager.cc:1918] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:29.432811 14152 log.cc:1196] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/a78afef7b421437cb7b3fe641f10e904
I20260807 18:38:29.433210 14152 ts_tablet_manager.cc:1939] T a78afef7b421437cb7b3fe641f10e904 P c4100644d4c74d5491fed65eb12df9bf: Deleting consensus metadata
I20260807 18:38:29.433827 13313 catalog_manager.cc:4841] TS a128dfadfd4747bcabe956f39dc82b68 (127.6.138.66:41877): tablet a78afef7b421437cb7b3fe641f10e904 (table test-table [id=4daa6d22c536472bb3029e48731e53e9]) successfully deleted
I20260807 18:38:29.434386 13312 catalog_manager.cc:4841] TS c4100644d4c74d5491fed65eb12df9bf (127.6.138.67:40865): tablet a78afef7b421437cb7b3fe641f10e904 (table test-table [id=4daa6d22c536472bb3029e48731e53e9]) successfully deleted
W20260807 18:38:29.443243 13310 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:35691 (error 108)
I20260807 18:38:29.443481 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 13645
I20260807 18:38:29.460121 14106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.446863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:35691 (local address 127.0.0.1:33728)
0807 18:38:29.459854 (+ 12991us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:29.459998 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:35691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12837}
W20260807 18:38:29.462095 13310 catalog_manager.cc:4584] TS 3575020d19b0490db85819b9b477e783 (127.6.138.65:35691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a78afef7b421437cb7b3fe641f10e904: Network error: Client connection negotiation failed: client connection to 127.6.138.65:35691: connect: Connection refused (error 111)
I20260807 18:38:29.467748 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 13860
I20260807 18:38:29.483655 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 13284
2026-08-07T18:38:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2668 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T18:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:29Z Disabled control of system clock
I20260807 18:38:29.523943 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:39955
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:45717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:39955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:38:29.638228 14158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:29.638522 14158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:29.638561 14158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:29.642499 14158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:29.642565 14158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:29.642581 14158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:29.642598 14158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:29.642613 14158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:29.648075 14158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:45717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:39955
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:39955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:29.649290 14158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:29.650799 14158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:29.659889 14168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:29.660089 14158 server_base.cc:1034] running on GCE node
W20260807 18:38:29.659914 14171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:29.659889 14169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:29.660908 14158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:29.662117 14158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:29.663296 14158 hybrid_clock.cc:648] HybridClock initialized: now 1786127909663262 us; error 45 us; skew 500 ppm
I20260807 18:38:29.665221 14158 webserver.cc:460] Webserver started at http://127.6.138.126:36071/ using document root <none> and password file <none>
I20260807 18:38:29.665828 14158 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:29.665892 14158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:29.666134 14158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:29.668324 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "1dcc4ab9fe4b408580cd6e734396247f"
format_stamp: "Formatted at 2026-08-07 18:38:29 on dist-test-slave-2d19"
server_key: "c338f2ccfd230e1517021f79fb686da5"
server_key_iv: "62800efc8381c6a624ff5d6362fde910"
server_key_version: "encryptionkey@0"
I20260807 18:38:29.668937 14158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "1dcc4ab9fe4b408580cd6e734396247f"
format_stamp: "Formatted at 2026-08-07 18:38:29 on dist-test-slave-2d19"
server_key: "c338f2ccfd230e1517021f79fb686da5"
server_key_iv: "62800efc8381c6a624ff5d6362fde910"
server_key_version: "encryptionkey@0"
I20260807 18:38:29.672752 14158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:38:29.675288 14179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:29.676648 14158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:29.676815 14158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "1dcc4ab9fe4b408580cd6e734396247f"
format_stamp: "Formatted at 2026-08-07 18:38:29 on dist-test-slave-2d19"
server_key: "c338f2ccfd230e1517021f79fb686da5"
server_key_iv: "62800efc8381c6a624ff5d6362fde910"
server_key_version: "encryptionkey@0"
I20260807 18:38:29.676944 14158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:29.681558 14158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:29.682158 14158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:29.682344 14158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:29.689837 14158 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:39955
I20260807 18:38:29.689873 14274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:39955 every 8 connection(s)
I20260807 18:38:29.690804 14158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:29.694192 14275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:29.699800 14275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f: Bootstrap starting.
I20260807 18:38:29.700062 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 14158
I20260807 18:38:29.700198 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:29.700503 6697 external_mini_cluster.cc:1428] Setting key e912d8e6d709243f3d283553d142478f
I20260807 18:38:29.702592 14275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:29.703351 14275 log.cc:824] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:29.705796 14275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f: No bootstrap required, opened a new log
I20260807 18:38:29.706390 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.701874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.6.138.126:39955)
0807 18:38:29.702554 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.702569 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.702603 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:38:29.703287 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.703305 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.703735 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.704120 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:29.704989 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:29.704993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:29.705005 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:29.705015 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:29.705071 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:29.705087 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:29.705176 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.705326 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.705625 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":151,"threads_started":1}
I20260807 18:38:29.709789 14275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc4ab9fe4b408580cd6e734396247f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39955 } }
I20260807 18:38:29.710037 14275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:29.710134 14275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcc4ab9fe4b408580cd6e734396247f, State: Initialized, Role: FOLLOWER
I20260807 18:38:29.710594 14275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc4ab9fe4b408580cd6e734396247f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39955 } }
I20260807 18:38:29.710754 14275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:29.710841 14275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:29.710942 14275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:29.711848 14275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc4ab9fe4b408580cd6e734396247f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39955 } }
I20260807 18:38:29.712251 14275 leader_election.cc:302] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dcc4ab9fe4b408580cd6e734396247f; no voters:
I20260807 18:38:29.712657 14275 leader_election.cc:288] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:29.712733 14282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:29.712944 14282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [term 1 LEADER]: Becoming Leader. State: Replica: 1dcc4ab9fe4b408580cd6e734396247f, State: Running, Role: LEADER
I20260807 18:38:29.713243 14282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc4ab9fe4b408580cd6e734396247f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39955 } }
I20260807 18:38:29.713974 14275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:29.714985 14284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dcc4ab9fe4b408580cd6e734396247f. Latest consensus state: current_term: 1 leader_uuid: "1dcc4ab9fe4b408580cd6e734396247f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc4ab9fe4b408580cd6e734396247f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39955 } } }
I20260807 18:38:29.715121 14284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:29.715389 14285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dcc4ab9fe4b408580cd6e734396247f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc4ab9fe4b408580cd6e734396247f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39955 } } }
I20260807 18:38:29.715505 14285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:29.715780 14294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:29.718951 14294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:29.724805 14294 catalog_manager.cc:1347] Generated new cluster ID: c195a19fc8a649b78c613794e87fc689
I20260807 18:38:29.724888 14294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:29.739702 14294 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:29.740083 14294 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:29.747360 14294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f: Generated new TSK 0
I20260807 18:38:29.748056 14294 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:38:29.754209 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39955
--builtin_ntp_servers=127.6.138.84:45717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:38:29.898381 14312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:29.898656 14312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:29.898698 14312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:29.903007 14312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:29.903069 14312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:29.903129 14312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:29.907943 14312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:45717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:29.909164 14312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:29.910619 14312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:29.921319 14322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:29.921324 14319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:29.921494 14312 server_base.cc:1034] running on GCE node
W20260807 18:38:29.921319 14318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:29.922200 14312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:29.923008 14312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:29.924220 14312 hybrid_clock.cc:648] HybridClock initialized: now 1786127909924197 us; error 32 us; skew 500 ppm
I20260807 18:38:29.926337 14312 webserver.cc:460] Webserver started at http://127.6.138.65:37869/ using document root <none> and password file <none>
I20260807 18:38:29.926911 14312 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:29.926975 14312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:29.927155 14312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:29.929527 14312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "915a1ee9ecf048a89594708e7e73a61e"
format_stamp: "Formatted at 2026-08-07 18:38:29 on dist-test-slave-2d19"
server_key: "806656fb95a766b2b52fad802d654c9a"
server_key_iv: "4404a22a72e24ea2abefabfc5893a1e6"
server_key_version: "encryptionkey@0"
I20260807 18:38:29.930055 14312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "915a1ee9ecf048a89594708e7e73a61e"
format_stamp: "Formatted at 2026-08-07 18:38:29 on dist-test-slave-2d19"
server_key: "806656fb95a766b2b52fad802d654c9a"
server_key_iv: "4404a22a72e24ea2abefabfc5893a1e6"
server_key_version: "encryptionkey@0"
I20260807 18:38:29.934022 14312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 18:38:29.936798 14328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:29.938216 14312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:29.938331 14312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "915a1ee9ecf048a89594708e7e73a61e"
format_stamp: "Formatted at 2026-08-07 18:38:29 on dist-test-slave-2d19"
server_key: "806656fb95a766b2b52fad802d654c9a"
server_key_iv: "4404a22a72e24ea2abefabfc5893a1e6"
server_key_version: "encryptionkey@0"
I20260807 18:38:29.938431 14312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:29.943394 14312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:29.944013 14312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:29.944176 14312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:29.944816 14312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:29.945756 14312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:29.945806 14312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:29.945847 14312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:29.945868 14312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:29.954725 14312 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:36613
I20260807 18:38:29.954751 14503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:36613 every 8 connection(s)
I20260807 18:38:29.955720 14312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:29.962335 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.957773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:33275 (local address 127.6.138.126:39955)
0807 18:38:29.957922 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:29.957926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:29.959141 (+ 1215us) server_negotiation.cc:407] Connection header received
0807 18:38:29.960630 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:29.960633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:29.960673 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:29.960758 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:29.961889 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:29.961892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:29.961894 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:29.961907 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:29.961928 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:29.961937 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:29.962033 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:29.962177 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 18:38:29.962219 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:38:29.962816 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 14312
I20260807 18:38:29.962920 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:29.963253 6697 external_mini_cluster.cc:1428] Setting key aa4c7cd1bf8d4c989f0587aa074f66b0
I20260807 18:38:29.963320 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:29.958261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39955 (local address 127.6.138.65:33275)
0807 18:38:29.958964 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:29.959006 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:29.960399 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:29.960930 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:29.960938 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:29.961026 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:29.961616 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:38:29.961617 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:29.961658 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:38:29.961725 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:29.962034 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:29.962042 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:29.962046 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 18:38:29.962186 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 18:38:29.962379 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":134,"threads_started":1}
I20260807 18:38:29.964700 14504 heartbeater.cc:347] Connected to a master server at 127.6.138.126:39955
I20260807 18:38:29.964951 14504 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:29.965366 14504 heartbeater.cc:511] Master 127.6.138.126:39955 requested a full tablet report, sending...
I20260807 18:38:29.965641 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39955
--builtin_ntp_servers=127.6.138.84:45717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:38:29.966964 14203 ts_manager.cc:194] Registered new tserver with Master: 915a1ee9ecf048a89594708e7e73a61e (127.6.138.65:36613)
W20260807 18:38:30.119118 14511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:30.119400 14511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:30.119495 14511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:30.123927 14511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:30.123991 14511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:30.124053 14511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:30.129046 14511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:45717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:30.130290 14511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:30.131752 14511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:30.143213 14521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:30.143213 14517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:30.143550 14511 server_base.cc:1034] running on GCE node
W20260807 18:38:30.143661 14523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:30.144353 14511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:30.145324 14511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:30.146549 14511 hybrid_clock.cc:648] HybridClock initialized: now 1786127910146524 us; error 38 us; skew 500 ppm
I20260807 18:38:30.150827 14511 webserver.cc:460] Webserver started at http://127.6.138.66:44439/ using document root <none> and password file <none>
I20260807 18:38:30.151645 14511 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:30.151744 14511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:30.152082 14511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:30.154294 14511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "7c5685f0de63450083c57d0f94a32137"
format_stamp: "Formatted at 2026-08-07 18:38:30 on dist-test-slave-2d19"
server_key: "3f93738edea2b0eb3c7372b2241a0a0d"
server_key_iv: "376ff36fcdece3167023f59f8fa02a0c"
server_key_version: "encryptionkey@0"
I20260807 18:38:30.154825 14511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "7c5685f0de63450083c57d0f94a32137"
format_stamp: "Formatted at 2026-08-07 18:38:30 on dist-test-slave-2d19"
server_key: "3f93738edea2b0eb3c7372b2241a0a0d"
server_key_iv: "376ff36fcdece3167023f59f8fa02a0c"
server_key_version: "encryptionkey@0"
I20260807 18:38:30.158826 14511 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:30.161484 14529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:30.162875 14511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:38:30.163077 14511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "7c5685f0de63450083c57d0f94a32137"
format_stamp: "Formatted at 2026-08-07 18:38:30 on dist-test-slave-2d19"
server_key: "3f93738edea2b0eb3c7372b2241a0a0d"
server_key_iv: "376ff36fcdece3167023f59f8fa02a0c"
server_key_version: "encryptionkey@0"
I20260807 18:38:30.163214 14511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:30.168020 14511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:30.168716 14511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:30.168915 14511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:30.169495 14511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:30.170446 14511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:30.170531 14511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:30.170603 14511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:30.170639 14511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:30.180181 14511 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:33281
I20260807 18:38:30.180230 14700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:33281 every 8 connection(s)
I20260807 18:38:30.181365 14511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:30.187799 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 14511
I20260807 18:38:30.187971 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:30.188205 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.183461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:53825 (local address 127.6.138.126:39955)
0807 18:38:30.183621 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.183625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.184918 (+ 1293us) server_negotiation.cc:407] Connection header received
0807 18:38:30.186263 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.186266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.186305 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.186395 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.187589 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.187596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.187601 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.187616 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.187643 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.187658 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.187778 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.187993 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.188061 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:38:30.188369 6697 external_mini_cluster.cc:1428] Setting key 15b959a4f4889ac1165958980e302027
I20260807 18:38:30.189193 14703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.183900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39955 (local address 127.6.138.66:53825)
0807 18:38:30.184760 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:30.184804 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:30.186036 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:30.186584 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:30.186592 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:30.186683 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:30.187282 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:38:30.187284 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:30.187325 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:38:30.187396 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:30.187766 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:30.187774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:30.187779 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:38:30.187957 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 18:38:30.188218 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":128,"threads_started":1}
I20260807 18:38:30.190426 14701 heartbeater.cc:347] Connected to a master server at 127.6.138.126:39955
I20260807 18:38:30.190665 14701 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:30.190730 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39955
--builtin_ntp_servers=127.6.138.84:45717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:38:30.191157 14701 heartbeater.cc:511] Master 127.6.138.126:39955 requested a full tablet report, sending...
I20260807 18:38:30.192221 14203 ts_manager.cc:194] Registered new tserver with Master: 7c5685f0de63450083c57d0f94a32137 (127.6.138.66:33281)
W20260807 18:38:30.309051 14704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:30.309324 14704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:30.309365 14704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:30.313436 14704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:30.313499 14704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:30.313556 14704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:30.318560 14704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:45717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:30.319633 14704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:30.321044 14704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:30.330663 14713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:30.330853 14704 server_base.cc:1034] running on GCE node
W20260807 18:38:30.330694 14714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:30.330694 14716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:30.331384 14704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:30.332037 14704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:30.333209 14704 hybrid_clock.cc:648] HybridClock initialized: now 1786127910333171 us; error 48 us; skew 500 ppm
I20260807 18:38:30.335034 14704 webserver.cc:460] Webserver started at http://127.6.138.67:36377/ using document root <none> and password file <none>
I20260807 18:38:30.335668 14704 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:30.335734 14704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:30.335968 14704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:30.338347 14704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "3260bcfe292b41e2a66689770108e638"
format_stamp: "Formatted at 2026-08-07 18:38:30 on dist-test-slave-2d19"
server_key: "4073eb3df281aeb51befb85c0b8137fe"
server_key_iv: "9c34f581c871e528d11beb4572115b37"
server_key_version: "encryptionkey@0"
I20260807 18:38:30.338951 14704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "3260bcfe292b41e2a66689770108e638"
format_stamp: "Formatted at 2026-08-07 18:38:30 on dist-test-slave-2d19"
server_key: "4073eb3df281aeb51befb85c0b8137fe"
server_key_iv: "9c34f581c871e528d11beb4572115b37"
server_key_version: "encryptionkey@0"
I20260807 18:38:30.343218 14704 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:38:30.346035 14727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:30.347584 14704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:38:30.347734 14704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "3260bcfe292b41e2a66689770108e638"
format_stamp: "Formatted at 2026-08-07 18:38:30 on dist-test-slave-2d19"
server_key: "4073eb3df281aeb51befb85c0b8137fe"
server_key_iv: "9c34f581c871e528d11beb4572115b37"
server_key_version: "encryptionkey@0"
I20260807 18:38:30.347883 14704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:30.352867 14704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:30.353516 14704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:30.353724 14704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:30.354322 14704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:30.355288 14704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:30.355378 14704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:30.355453 14704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:30.355489 14704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:30.365006 14704 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:44087
I20260807 18:38:30.365027 14904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:44087 every 8 connection(s)
I20260807 18:38:30.365996 14704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:30.368439 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 14704
I20260807 18:38:30.368557 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:30.368891 6697 external_mini_cluster.cc:1428] Setting key 6a59c117d8ab849f31c5927621ab1dd4
I20260807 18:38:30.373565 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.368122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:45429 (local address 127.6.138.126:39955)
0807 18:38:30.368299 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.368306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.369798 (+ 1492us) server_negotiation.cc:407] Connection header received
0807 18:38:30.371656 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.371659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.371707 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.371788 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.373074 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.373078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.373080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.373097 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.373116 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.373124 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.373224 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.373379 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.373429 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:38:30.374487 14909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.368716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39955 (local address 127.6.138.67:45429)
0807 18:38:30.369604 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:30.369650 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:30.371385 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:30.372022 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:30.372029 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:30.372128 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:30.372794 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:38:30.372795 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:30.372838 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:38:30.372912 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:30.373212 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:30.373221 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:30.373228 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:38:30.373431 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 18:38:30.373710 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":154,"threads_started":1}
I20260807 18:38:30.375653 14907 heartbeater.cc:347] Connected to a master server at 127.6.138.126:39955
I20260807 18:38:30.375878 14907 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:30.376354 14907 heartbeater.cc:511] Master 127.6.138.126:39955 requested a full tablet report, sending...
I20260807 18:38:30.377411 14203 ts_manager.cc:194] Registered new tserver with Master: 3260bcfe292b41e2a66689770108e638 (127.6.138.67:44087)
I20260807 18:38:30.383335 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:38:30.387518 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.385541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33008 (local address 127.6.138.126:39955)
0807 18:38:30.385764 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.385769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.385967 (+ 198us) server_negotiation.cc:407] Connection header received
0807 18:38:30.386141 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.386144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.386185 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.386275 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.387081 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.387085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.387087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.387100 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.387122 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.387137 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.387232 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.387310 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.387355 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260807 18:38:30.390909 14203 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:30.394225 14203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:30.406986 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.403577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.6.138.65:36613)
0807 18:38:30.404399 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.404414 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.404443 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:38:30.404501 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.404509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.404731 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.405492 (+ 761us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.406304 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.406310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.406323 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.406333 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.406377 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.406411 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.406521 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.406712 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.406797 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":140,"threads_started":1}
I20260807 18:38:30.407171 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.403435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:36613 (local address 127.0.0.1:42664)
0807 18:38:30.403835 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:30.403891 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:30.404001 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:30.404920 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:30.404926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:30.405175 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:30.405805 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:38:30.405807 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:30.405831 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:38:30.405904 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:30.406521 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:30.406533 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:30.406539 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:38:30.406665 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:38:30.407013 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":103,"threads_started":1}
I20260807 18:38:30.407800 14923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.404024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:33281 (local address 127.0.0.1:54280)
0807 18:38:30.404354 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:30.404368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:30.404448 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:30.405735 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:30.405739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:30.405755 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:30.406295 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:38:30.406297 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:30.406307 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:38:30.406332 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:30.407556 (+ 1224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:30.407561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:30.407565 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 18:38:30.407628 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:38:30.407679 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":149,"threads_started":1}
I20260807 18:38:30.407852 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.404525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.6.138.67:44087)
0807 18:38:30.405035 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.405049 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.405302 (+ 253us) server_negotiation.cc:407] Connection header received
0807 18:38:30.405496 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.405502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.405678 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.405804 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.407211 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.407220 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.407231 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.407247 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.407291 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.407335 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.407419 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.407618 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.407688 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":168,"threads_started":1}
I20260807 18:38:30.408010 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.404523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.6.138.66:33281)
0807 18:38:30.404925 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.404939 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.404961 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:38:30.405019 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.405026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.405243 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.406162 (+ 919us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.406978 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.406985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.406998 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.407012 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.407424 (+ 412us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.407470 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.407592 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.407650 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.407722 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":156,"threads_started":1}
I20260807 18:38:30.408161 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.404859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:44087 (local address 127.0.0.1:60722)
0807 18:38:30.405187 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:30.405211 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:30.405360 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:30.406545 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:30.406548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:30.406562 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:30.407020 (+ 458us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:38:30.407022 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:30.407032 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:38:30.407058 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:30.407497 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:30.407500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:30.407503 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 18:38:30.408005 (+ 502us) client_negotiation.cc:240] Negotiation successful
0807 18:38:30.408043 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":82,"threads_started":1}
I20260807 18:38:30.410600 14601 tablet_service.cc:1467] Processing CreateTablet for tablet 26df8db6eb9e44fa9913cccf363fa917 (DEFAULT_TABLE table=test-table [id=a35612617e084ca6aff5a699fab4f0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:30.410573 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 26df8db6eb9e44fa9913cccf363fa917 (DEFAULT_TABLE table=test-table [id=a35612617e084ca6aff5a699fab4f0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:30.410759 14799 tablet_service.cc:1467] Processing CreateTablet for tablet 26df8db6eb9e44fa9913cccf363fa917 (DEFAULT_TABLE table=test-table [id=a35612617e084ca6aff5a699fab4f0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:30.411779 14601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26df8db6eb9e44fa9913cccf363fa917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:30.411779 14398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26df8db6eb9e44fa9913cccf363fa917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:30.411810 14799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26df8db6eb9e44fa9913cccf363fa917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:30.418653 14929 tablet_bootstrap.cc:492] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: Bootstrap starting.
I20260807 18:38:30.419948 14930 tablet_bootstrap.cc:492] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: Bootstrap starting.
I20260807 18:38:30.421243 14929 tablet_bootstrap.cc:654] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:30.422029 14930 tablet_bootstrap.cc:654] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:30.422015 14929 log.cc:824] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:30.422700 14931 tablet_bootstrap.cc:492] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e: Bootstrap starting.
I20260807 18:38:30.422905 14930 log.cc:824] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:30.424283 14929 tablet_bootstrap.cc:492] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: No bootstrap required, opened a new log
I20260807 18:38:30.424490 14929 ts_tablet_manager.cc:1397] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:38:30.424882 14931 tablet_bootstrap.cc:654] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:30.425038 14930 tablet_bootstrap.cc:492] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: No bootstrap required, opened a new log
I20260807 18:38:30.425222 14930 ts_tablet_manager.cc:1397] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:38:30.425678 14931 log.cc:824] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:30.427556 14931 tablet_bootstrap.cc:492] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e: No bootstrap required, opened a new log
I20260807 18:38:30.427765 14931 ts_tablet_manager.cc:1397] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:38:30.429441 14929 raft_consensus.cc:348] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } }
I20260807 18:38:30.429750 14929 raft_consensus.cc:374] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:30.429813 14929 raft_consensus.cc:729] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3260bcfe292b41e2a66689770108e638, State: Initialized, Role: FOLLOWER
I20260807 18:38:30.429826 14930 raft_consensus.cc:348] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } }
I20260807 18:38:30.430063 14930 raft_consensus.cc:374] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:30.430112 14930 raft_consensus.cc:729] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c5685f0de63450083c57d0f94a32137, State: Initialized, Role: FOLLOWER
I20260807 18:38:30.430322 14929 consensus_queue.cc:260] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } }
I20260807 18:38:30.430567 14930 consensus_queue.cc:260] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } }
I20260807 18:38:30.431181 14907 heartbeater.cc:503] Master 127.6.138.126:39955 was elected leader, sending a full tablet report...
I20260807 18:38:30.431504 14930 ts_tablet_manager.cc:1428] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 18:38:30.431892 14701 heartbeater.cc:503] Master 127.6.138.126:39955 was elected leader, sending a full tablet report...
I20260807 18:38:30.432057 14931 raft_consensus.cc:348] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } }
I20260807 18:38:30.432329 14931 raft_consensus.cc:374] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:30.432387 14931 raft_consensus.cc:729] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915a1ee9ecf048a89594708e7e73a61e, State: Initialized, Role: FOLLOWER
I20260807 18:38:30.432451 14929 ts_tablet_manager.cc:1428] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260807 18:38:30.432523 14702 tablet.cc:2369] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:30.432852 14931 consensus_queue.cc:260] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } }
I20260807 18:38:30.433830 14931 ts_tablet_manager.cc:1428] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:38:30.434161 14504 heartbeater.cc:503] Master 127.6.138.126:39955 was elected leader, sending a full tablet report...
I20260807 18:38:30.450775 14937 raft_consensus.cc:482] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:30.451012 14937 raft_consensus.cc:504] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } }
I20260807 18:38:30.452621 14937 leader_election.cc:288] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 915a1ee9ecf048a89594708e7e73a61e (127.6.138.65:36613), 7c5685f0de63450083c57d0f94a32137 (127.6.138.66:33281)
I20260807 18:38:30.454766 14909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.452809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:36613 (local address 127.6.138.67:42791)
0807 18:38:30.452969 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:30.452987 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:30.453083 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:30.453417 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:30.453423 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:30.453445 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:30.454028 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:38:30.454031 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:30.454044 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:38:30.454103 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:30.454479 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:30.454482 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:30.454486 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 18:38:30.454576 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:38:30.454618 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:38:30.454802 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.453025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:42791 (local address 127.6.138.65:36613)
0807 18:38:30.453168 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.453174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.453191 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:38:30.453241 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.453245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.453283 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.453379 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.454285 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.454291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.454294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.454311 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.454335 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.454347 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.454461 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.454588 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.454649 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:38:30.455008 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.452898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:33281 (local address 127.6.138.67:39607)
0807 18:38:30.453489 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:30.453504 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:30.453594 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:30.453872 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:30.453875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:30.453892 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:30.454332 (+ 440us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:38:30.454334 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:30.454345 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:38:30.454372 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:30.454665 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:30.454668 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:30.454671 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 18:38:30.454749 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 18:38:30.454782 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":145,"threads_started":1}
I20260807 18:38:30.455003 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.453370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:39607 (local address 127.6.138.66:33281)
0807 18:38:30.453546 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.453554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.453573 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:38:30.453705 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.453708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.453749 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.453843 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.454508 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.454514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.454520 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.454537 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.454563 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.454574 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.454660 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.454761 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.454804 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:38:30.455639 14633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26df8db6eb9e44fa9913cccf363fa917" candidate_uuid: "3260bcfe292b41e2a66689770108e638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c5685f0de63450083c57d0f94a32137" is_pre_election: true
I20260807 18:38:30.455710 14428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26df8db6eb9e44fa9913cccf363fa917" candidate_uuid: "3260bcfe292b41e2a66689770108e638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a1ee9ecf048a89594708e7e73a61e" is_pre_election: true
I20260807 18:38:30.455971 14633 raft_consensus.cc:2454] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3260bcfe292b41e2a66689770108e638 in term 0.
I20260807 18:38:30.456008 14428 raft_consensus.cc:2454] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3260bcfe292b41e2a66689770108e638 in term 0.
I20260807 18:38:30.456467 14729 leader_election.cc:302] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3260bcfe292b41e2a66689770108e638, 7c5685f0de63450083c57d0f94a32137; no voters:
I20260807 18:38:30.456748 14937 raft_consensus.cc:2781] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:30.456833 14937 raft_consensus.cc:482] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:30.456892 14937 raft_consensus.cc:3037] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:38:30.457067 14508 tablet.cc:2369] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:30.458089 14937 raft_consensus.cc:504] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } }
I20260807 18:38:30.458506 14937 leader_election.cc:288] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [CANDIDATE]: Term 1 election: Requested vote from peers 915a1ee9ecf048a89594708e7e73a61e (127.6.138.65:36613), 7c5685f0de63450083c57d0f94a32137 (127.6.138.66:33281)
I20260807 18:38:30.458843 14428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26df8db6eb9e44fa9913cccf363fa917" candidate_uuid: "3260bcfe292b41e2a66689770108e638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915a1ee9ecf048a89594708e7e73a61e"
I20260807 18:38:30.458967 14428 raft_consensus.cc:3037] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:30.459010 14633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26df8db6eb9e44fa9913cccf363fa917" candidate_uuid: "3260bcfe292b41e2a66689770108e638" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c5685f0de63450083c57d0f94a32137"
I20260807 18:38:30.459182 14633 raft_consensus.cc:3037] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:30.459952 14428 raft_consensus.cc:2454] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3260bcfe292b41e2a66689770108e638 in term 1.
I20260807 18:38:30.460237 14728 leader_election.cc:302] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3260bcfe292b41e2a66689770108e638, 915a1ee9ecf048a89594708e7e73a61e; no voters:
I20260807 18:38:30.460494 14937 raft_consensus.cc:2781] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:30.460649 14633 raft_consensus.cc:2454] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3260bcfe292b41e2a66689770108e638 in term 1.
I20260807 18:38:30.460737 14937 raft_consensus.cc:686] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 1 LEADER]: Becoming Leader. State: Replica: 3260bcfe292b41e2a66689770108e638, State: Running, Role: LEADER
I20260807 18:38:30.461213 14937 consensus_queue.cc:237] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } }
I20260807 18:38:30.464457 14204 catalog_manager.cc:5495] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3260bcfe292b41e2a66689770108e638 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "3260bcfe292b41e2a66689770108e638" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3260bcfe292b41e2a66689770108e638" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 } health_report { overall_health: UNKNOWN } } }
I20260807 18:38:30.488447 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.486860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.6.138.67:44087)
0807 18:38:30.487034 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.487038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.487054 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:30.487132 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.487135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.487194 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.487271 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.487990 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.487993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.487995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.488007 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.488027 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.488035 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.488121 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.488183 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.488219 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:38:30.494683 14633 raft_consensus.cc:1261] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [term 1 FOLLOWER]: Refusing update from remote peer 3260bcfe292b41e2a66689770108e638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:30.494701 14428 raft_consensus.cc:1261] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e [term 1 FOLLOWER]: Refusing update from remote peer 3260bcfe292b41e2a66689770108e638: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:30.495544 14937 consensus_queue.cc:1035] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915a1ee9ecf048a89594708e7e73a61e" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 36613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:30.495761 14944 consensus_queue.cc:1035] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c5685f0de63450083c57d0f94a32137" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 33281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:30.505188 14949 mvcc.cc:205] Tried to move back new op lower bound from 7315979921378447360 to 7315979921251401728. Current Snapshot: MvccSnapshot[applied={T|T < 7315979921378447360}]
I20260807 18:38:30.512109 14203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:30.512331 14203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:30.515017 14203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dcc4ab9fe4b408580cd6e734396247f: Sending DeleteTablet for 3 replicas of tablet 26df8db6eb9e44fa9913cccf363fa917
I20260807 18:38:30.515954 14799 tablet_service.cc:1514] Processing DeleteTablet for tablet 26df8db6eb9e44fa9913cccf363fa917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:30 UTC) from {username='slave'} at 127.0.0.1:60722
I20260807 18:38:30.515957 14601 tablet_service.cc:1514] Processing DeleteTablet for tablet 26df8db6eb9e44fa9913cccf363fa917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:30 UTC) from {username='slave'} at 127.0.0.1:54280
I20260807 18:38:30.516511 14965 tablet_replica.cc:330] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: stopping tablet replica
I20260807 18:38:30.516492 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 26df8db6eb9e44fa9913cccf363fa917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:30 UTC) from {username='slave'} at 127.0.0.1:42664
I20260807 18:38:30.516860 14965 raft_consensus.cc:2229] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:30.517092 14968 tablet_replica.cc:330] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e: stopping tablet replica
I20260807 18:38:30.517247 14965 raft_consensus.cc:2258] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:30.517493 14968 raft_consensus.cc:2229] T 26df8db6eb9e44fa9913cccf363fa917 P 915a1ee9ecf048a89594708e7e73a61e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:30.517632 14966 tablet_replica.cc:330] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: stopping tablet replica
I20260807 18:38:30.517696 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 14312
I20260807 18:38:30.518364 14966 raft_consensus.cc:2229] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:30.518664 14966 raft_consensus.cc:2258] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:30.522020 14966 ts_tablet_manager.cc:1905] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:30.525544 14966 ts_tablet_manager.cc:1918] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:30.525642 14966 log.cc:1196] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/26df8db6eb9e44fa9913cccf363fa917
I20260807 18:38:30.525939 14966 ts_tablet_manager.cc:1939] T 26df8db6eb9e44fa9913cccf363fa917 P 7c5685f0de63450083c57d0f94a32137: Deleting consensus metadata
I20260807 18:38:30.527868 14181 catalog_manager.cc:4841] TS 7c5685f0de63450083c57d0f94a32137 (127.6.138.66:33281): tablet 26df8db6eb9e44fa9913cccf363fa917 (table test-table [id=a35612617e084ca6aff5a699fab4f0da]) successfully deleted
I20260807 18:38:30.528640 14965 ts_tablet_manager.cc:1905] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:30.532323 14965 ts_tablet_manager.cc:1918] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:30.532441 14965 log.cc:1196] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/26df8db6eb9e44fa9913cccf363fa917
I20260807 18:38:30.532814 14965 ts_tablet_manager.cc:1939] T 26df8db6eb9e44fa9913cccf363fa917 P 3260bcfe292b41e2a66689770108e638: Deleting consensus metadata
I20260807 18:38:30.534274 14180 catalog_manager.cc:4841] TS 3260bcfe292b41e2a66689770108e638 (127.6.138.67:44087): tablet 26df8db6eb9e44fa9913cccf363fa917 (table test-table [id=a35612617e084ca6aff5a699fab4f0da]) successfully deleted
W20260807 18:38:30.535534 14180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:36613 (error 108)
I20260807 18:38:30.536530 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 14511
I20260807 18:38:30.536787 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.536411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:36613 (local address 127.0.0.1:42680)
0807 18:38:30.536586 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:30.536677 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:36613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:38:30.542393 14180 catalog_manager.cc:4584] TS 915a1ee9ecf048a89594708e7e73a61e (127.6.138.65:36613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26df8db6eb9e44fa9913cccf363fa917: Network error: Client connection negotiation failed: client connection to 127.6.138.65:36613: connect: Connection refused (error 111)
I20260807 18:38:30.564129 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 14704
I20260807 18:38:30.577848 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 14158
2026-08-07T18:38:30Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1092 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 18:38:30.607168 6697 mini_oidc.cc:152] Starting JWKS server
I20260807 18:38:30.609196 6697 webserver.cc:460] Webserver started at https://127.0.0.1:39771/ using document root <none> and password file <none>
I20260807 18:38:30.609400 6697 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:38:30.609748 6697 webserver.cc:460] Webserver started at http://127.0.0.1:42011/ using document root <none> and password file <none>
2026-08-07T18:38:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:30Z Disabled control of system clock
I20260807 18:38:30.621025 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:45335
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:38549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:45335
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39771/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/testchainca.cert with env {}
W20260807 18:38:30.739076 14980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:30.739405 14980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:30.739475 14980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:30.743685 14980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:30.743810 14980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:30.743841 14980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:30.743880 14980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:30.743911 14980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:38:30.743942 14980 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:38:30.743981 14980 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39771/jwks.json
I20260807 18:38:30.749540 14980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:45335
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:45335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39771/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:30.750713 14980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:30.751807 14980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:30.761040 14995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:30.761191 14980 server_base.cc:1034] running on GCE node
W20260807 18:38:30.761124 14993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:30.761078 14991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:30.761919 14980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:30.763055 14980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:30.764225 14980 hybrid_clock.cc:648] HybridClock initialized: now 1786127910764205 us; error 32 us; skew 500 ppm
I20260807 18:38:30.766165 14980 webserver.cc:460] Webserver started at http://127.6.138.126:45559/ using document root <none> and password file <none>
I20260807 18:38:30.766772 14980 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:30.766836 14980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:30.767083 14980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:30.769297 14980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "349192a7caae422ebddb8b1f7373857f"
format_stamp: "Formatted at 2026-08-07 18:38:30 on dist-test-slave-2d19"
server_key: "232910f9200cb57277bd6fab7ad9cf57"
server_key_iv: "8cb1a2121de0c356528e5239c8d2d55a"
server_key_version: "encryptionkey@0"
I20260807 18:38:30.769881 14980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "349192a7caae422ebddb8b1f7373857f"
format_stamp: "Formatted at 2026-08-07 18:38:30 on dist-test-slave-2d19"
server_key: "232910f9200cb57277bd6fab7ad9cf57"
server_key_iv: "8cb1a2121de0c356528e5239c8d2d55a"
server_key_version: "encryptionkey@0"
I20260807 18:38:30.773713 14980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:38:30.776177 15003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:30.777674 14980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:38:30.777828 14980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "349192a7caae422ebddb8b1f7373857f"
format_stamp: "Formatted at 2026-08-07 18:38:30 on dist-test-slave-2d19"
server_key: "232910f9200cb57277bd6fab7ad9cf57"
server_key_iv: "8cb1a2121de0c356528e5239c8d2d55a"
server_key_version: "encryptionkey@0"
I20260807 18:38:30.777956 14980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:30.790086 14980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:30.790920 14980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:30.791150 14980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:30.799510 15090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:45335 every 8 connection(s)
I20260807 18:38:30.799532 14980 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:45335
I20260807 18:38:30.800765 14980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:30.804152 15091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:30.807207 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 14980
I20260807 18:38:30.807345 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:30.807654 6697 external_mini_cluster.cc:1428] Setting key 09033ad30a269f585d97458150f3e57d
I20260807 18:38:30.810839 15091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f: Bootstrap starting.
I20260807 18:38:30.813642 15091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:30.814571 15091 log.cc:824] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:30.817101 15091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f: No bootstrap required, opened a new log
I20260807 18:38:30.818152 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.809178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48424 (local address 127.6.138.126:45335)
0807 18:38:30.809979 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.809999 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.810040 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:38:30.810711 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.810728 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.811213 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.811527 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:30.811917 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:30.813024 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:30.814292 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:30.814845 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:30.815012 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:30.815917 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:30.815923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:30.815939 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:30.816493 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:30.816564 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:30.816599 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:30.816710 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:30.816900 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 18:38:30.817180 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":194,"threads_started":1}
I20260807 18:38:30.821130 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:38:30.821552 15091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349192a7caae422ebddb8b1f7373857f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45335 } }
I20260807 18:38:30.821805 15091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:30.821883 15091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349192a7caae422ebddb8b1f7373857f, State: Initialized, Role: FOLLOWER
I20260807 18:38:30.822371 15091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349192a7caae422ebddb8b1f7373857f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45335 } }
I20260807 18:38:30.822542 15091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:30.822633 15091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:30.822743 15091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:30.823719 15091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349192a7caae422ebddb8b1f7373857f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45335 } }
I20260807 18:38:30.824072 15091 leader_election.cc:302] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 349192a7caae422ebddb8b1f7373857f; no voters:
I20260807 18:38:30.824460 15091 leader_election.cc:288] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:30.824596 15100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:30.824841 15100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [term 1 LEADER]: Becoming Leader. State: Replica: 349192a7caae422ebddb8b1f7373857f, State: Running, Role: LEADER
I20260807 18:38:30.825203 15100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349192a7caae422ebddb8b1f7373857f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45335 } }
I20260807 18:38:30.825735 15091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:30.827075 15103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 349192a7caae422ebddb8b1f7373857f. Latest consensus state: current_term: 1 leader_uuid: "349192a7caae422ebddb8b1f7373857f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349192a7caae422ebddb8b1f7373857f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45335 } } }
I20260807 18:38:30.827212 15103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:30.827683 15101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "349192a7caae422ebddb8b1f7373857f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349192a7caae422ebddb8b1f7373857f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45335 } } }
I20260807 18:38:30.828424 15101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f [sys.catalog]: This master's current role is: LEADER
W20260807 18:38:30.830631 15128 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:38:30.830696 15128 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:38:30.830845 15117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:30.831560 15117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:30.837329 15117 catalog_manager.cc:1347] Generated new cluster ID: 46b734dea98d4d50882fcded4c9dfeaa
I20260807 18:38:30.837409 15117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:30.850621 15117 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:30.851629 15117 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:30.856231 15117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 349192a7caae422ebddb8b1f7373857f: Generated new TSK 0
I20260807 18:38:30.857056 15117 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:38:30.891489 15140 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 18:38:30.891697 15140 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:38:30.892351 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:30.873770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.6.138.126:45335)
0807 18:38:30.873933 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:30.873938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:30.874170 (+ 232us) server_negotiation.cc:407] Connection header received
0807 18:38:30.874400 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:30.874403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:30.874453 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:30.874543 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:38:30.874841 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:30.875383 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:30.876584 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:30.876942 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:30.877056 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:30.877407 (+ 351us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:38:30.891833 (+ 14426us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:38:30.892165 (+ 332us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48436: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 18:38:30.892428 15095 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48436: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 18:38:30.894021 15139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:30.873688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:45335 (local address 127.0.0.1:48436)
0807 18:38:30.874069 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:30.874086 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:30.874234 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:30.874591 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:30.874594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:30.874612 (+ 18us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:38:30.874728 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:30.874735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:30.875496 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:30.875500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:30.876455 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:30.876465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:30.877074 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:30.877077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:30.877142 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:30.877189 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:38:30.892111 (+ 14922us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:38:30.892397 (+ 286us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.138.126:45335: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":135,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.138.126:45335: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39771/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 18:38:30.894886 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 14980
2026-08-07T18:38:30Z chronyd exiting
I20260807 18:38:30.936537 6697 test_util.cc:175] -----------------------------------------------
I20260807 18:38:30.936645 6697 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127881318083-6697-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (331 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 18:38:30.937814 6697 mini_oidc.cc:152] Starting JWKS server
I20260807 18:38:30.939118 6697 webserver.cc:460] Webserver started at https://127.0.0.1:35669/ using document root <none> and password file <none>
I20260807 18:38:30.939204 6697 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:38:30.939419 6697 webserver.cc:460] Webserver started at http://127.0.0.1:35437/ using document root <none> and password file <none>
2026-08-07T18:38:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:30Z Disabled control of system clock
I20260807 18:38:30.950155 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:39225
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:39225
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35669/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/testchainca.cert with env {}
W20260807 18:38:31.065032 15151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:31.065382 15151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:31.065466 15151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:31.069621 15151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:31.069726 15151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:31.069757 15151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:31.069795 15151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:31.069825 15151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:38:31.069865 15151 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:38:31.069895 15151 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35669/jwks.json
I20260807 18:38:31.075639 15151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:39225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:39225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35669/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:31.076884 15151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:31.077971 15151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:31.087466 15158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:31.087538 15151 server_base.cc:1034] running on GCE node
W20260807 18:38:31.087438 15162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:31.087438 15160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:31.088440 15151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:31.089659 15151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:31.090940 15151 hybrid_clock.cc:648] HybridClock initialized: now 1786127911090926 us; error 34 us; skew 500 ppm
I20260807 18:38:31.093210 15151 webserver.cc:460] Webserver started at http://127.6.138.126:34255/ using document root <none> and password file <none>
I20260807 18:38:31.093889 15151 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:31.094002 15151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:31.094260 15151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:31.096750 15151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "e19256d810b045f0b32bfc3bf5f6c5cb"
format_stamp: "Formatted at 2026-08-07 18:38:31 on dist-test-slave-2d19"
server_key: "f63ed83f6321ce8abb4b3c49a5216a4f"
server_key_iv: "7b6987325bea30ade6848e1efe8bfce1"
server_key_version: "encryptionkey@0"
I20260807 18:38:31.097446 15151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "e19256d810b045f0b32bfc3bf5f6c5cb"
format_stamp: "Formatted at 2026-08-07 18:38:31 on dist-test-slave-2d19"
server_key: "f63ed83f6321ce8abb4b3c49a5216a4f"
server_key_iv: "7b6987325bea30ade6848e1efe8bfce1"
server_key_version: "encryptionkey@0"
I20260807 18:38:31.101686 15151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:38:31.104478 15173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:31.105923 15151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:31.106101 15151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "e19256d810b045f0b32bfc3bf5f6c5cb"
format_stamp: "Formatted at 2026-08-07 18:38:31 on dist-test-slave-2d19"
server_key: "f63ed83f6321ce8abb4b3c49a5216a4f"
server_key_iv: "7b6987325bea30ade6848e1efe8bfce1"
server_key_version: "encryptionkey@0"
I20260807 18:38:31.106253 15151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:31.170367 15151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:31.171238 15151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:31.171479 15151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:31.179137 15151 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:39225
I20260807 18:38:31.179160 15261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:39225 every 8 connection(s)
I20260807 18:38:31.180208 15151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:31.183545 15262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:31.187899 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 15151
I20260807 18:38:31.188045 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:31.188418 6697 external_mini_cluster.cc:1428] Setting key dc14f215490be4a0916116638f0b4065
I20260807 18:38:31.190901 15262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb: Bootstrap starting.
I20260807 18:38:31.193423 15262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:31.194221 15262 log.cc:824] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:31.196416 15262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb: No bootstrap required, opened a new log
I20260807 18:38:31.199040 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:31.190167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.6.138.126:39225)
0807 18:38:31.190951 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:31.190968 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:31.191040 (+ 72us) server_negotiation.cc:407] Connection header received
0807 18:38:31.191745 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:31.191761 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:31.192193 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:31.192622 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:31.193087 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.194185 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.195381 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.195855 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.196048 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:31.197009 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:31.197014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:31.197028 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:31.197428 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:31.197509 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:31.197531 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:31.197650 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:31.197815 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 18:38:31.198102 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":176,"threads_started":1}
I20260807 18:38:31.199635 15262 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19256d810b045f0b32bfc3bf5f6c5cb" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39225 } }
I20260807 18:38:31.199889 15262 raft_consensus.cc:374] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:31.199968 15262 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19256d810b045f0b32bfc3bf5f6c5cb, State: Initialized, Role: FOLLOWER
I20260807 18:38:31.200453 15262 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19256d810b045f0b32bfc3bf5f6c5cb" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39225 } }
I20260807 18:38:31.200608 15262 raft_consensus.cc:388] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:31.200682 15262 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:31.200815 15262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:31.201869 15262 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19256d810b045f0b32bfc3bf5f6c5cb" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39225 } }
I20260807 18:38:31.202059 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:38:31.202255 15262 leader_election.cc:302] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e19256d810b045f0b32bfc3bf5f6c5cb; no voters:
I20260807 18:38:31.202605 15262 leader_election.cc:288] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:31.202677 15267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:31.202872 15267 raft_consensus.cc:686] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [term 1 LEADER]: Becoming Leader. State: Replica: e19256d810b045f0b32bfc3bf5f6c5cb, State: Running, Role: LEADER
I20260807 18:38:31.203236 15267 consensus_queue.cc:237] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19256d810b045f0b32bfc3bf5f6c5cb" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39225 } }
I20260807 18:38:31.203893 15262 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:31.204973 15268 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e19256d810b045f0b32bfc3bf5f6c5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19256d810b045f0b32bfc3bf5f6c5cb" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39225 } } }
I20260807 18:38:31.205338 15268 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:31.205238 15269 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e19256d810b045f0b32bfc3bf5f6c5cb. Latest consensus state: current_term: 1 leader_uuid: "e19256d810b045f0b32bfc3bf5f6c5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19256d810b045f0b32bfc3bf5f6c5cb" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39225 } } }
I20260807 18:38:31.205693 15269 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:31.206465 15282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:31.209263 15282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:31.214615 15282 catalog_manager.cc:1347] Generated new cluster ID: bf01eb7715164c839f1e632e333dbcac
I20260807 18:38:31.214684 15282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:31.250458 15282 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:31.251431 15282 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:31.255843 15282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e19256d810b045f0b32bfc3bf5f6c5cb: Generated new TSK 0
I20260807 18:38:31.256754 15282 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:38:31.295591 15302 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 18:38:31.295698 15302 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:38:31.296545 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:31.279770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58630 (local address 127.6.138.126:39225)
0807 18:38:31.279930 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:31.279935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:31.280110 (+ 175us) server_negotiation.cc:407] Connection header received
0807 18:38:31.280304 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:31.280307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:31.280375 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:31.280514 (+ 139us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:38:31.280776 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.281716 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.282703 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.283063 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.283170 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:31.283554 (+ 384us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:38:31.296045 (+ 12491us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:38:31.296387 (+ 342us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58630: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 18:38:31.296620 15265 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58630: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 18:38:31.296679 15301 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:31.279629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39225 (local address 127.0.0.1:58630)
0807 18:38:31.280005 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:31.280025 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:31.280139 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:31.280517 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:31.280519 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:31.280526 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:38:31.280653 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:31.280658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.281882 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.281886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:31.282549 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:31.282559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.283207 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.283212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:31.283308 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:31.283340 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:38:31.296314 (+ 12974us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:38:31.296473 (+ 159us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.138.126:39225: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":118,"threads_started":1}
I20260807 18:38:31.297683 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 15151
2026-08-07T18:38:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (410 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 18:38:31.348624 6697 mini_oidc.cc:152] Starting JWKS server
I20260807 18:38:31.349545 6697 webserver.cc:460] Webserver started at https://127.0.0.1:43307/ using document root <none> and password file <none>
I20260807 18:38:31.349669 6697 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:38:31.349946 6697 webserver.cc:460] Webserver started at http://127.0.0.1:35699/ using document root <none> and password file <none>
2026-08-07T18:38:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:31Z Disabled control of system clock
I20260807 18:38:31.360749 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:33193
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:33193
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43307/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:38:31.476531 15311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:31.476815 15311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:31.476855 15311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:31.480815 15311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:31.480870 15311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:31.480885 15311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:31.480901 15311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:31.480971 15311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:38:31.481002 15311 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:38:31.481025 15311 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43307/jwks.json
I20260807 18:38:31.486310 15311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:33193
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:33193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43307/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:31.487416 15311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:31.488880 15311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:31.497872 15324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:31.497887 15328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:31.497874 15325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:31.498265 15311 server_base.cc:1034] running on GCE node
I20260807 18:38:31.498907 15311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:31.500200 15311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:31.501432 15311 hybrid_clock.cc:648] HybridClock initialized: now 1786127911501398 us; error 48 us; skew 500 ppm
I20260807 18:38:31.503410 15311 webserver.cc:460] Webserver started at http://127.6.138.126:36163/ using document root <none> and password file <none>
I20260807 18:38:31.504034 15311 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:31.504134 15311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:31.504434 15311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:31.506567 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "347ed0f5489542b3b6f54c3173f8bbb2"
format_stamp: "Formatted at 2026-08-07 18:38:31 on dist-test-slave-2d19"
server_key: "05c3334a2b69b9883d1ef8d051a60f4e"
server_key_iv: "9d39c17003fe011cb7ec0faa8736e1e0"
server_key_version: "encryptionkey@0"
I20260807 18:38:31.507167 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "347ed0f5489542b3b6f54c3173f8bbb2"
format_stamp: "Formatted at 2026-08-07 18:38:31 on dist-test-slave-2d19"
server_key: "05c3334a2b69b9883d1ef8d051a60f4e"
server_key_iv: "9d39c17003fe011cb7ec0faa8736e1e0"
server_key_version: "encryptionkey@0"
I20260807 18:38:31.511037 15311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:38:31.513587 15343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:31.514940 15311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:31.515117 15311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "347ed0f5489542b3b6f54c3173f8bbb2"
format_stamp: "Formatted at 2026-08-07 18:38:31 on dist-test-slave-2d19"
server_key: "05c3334a2b69b9883d1ef8d051a60f4e"
server_key_iv: "9d39c17003fe011cb7ec0faa8736e1e0"
server_key_version: "encryptionkey@0"
I20260807 18:38:31.515244 15311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:31.539409 15311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:31.540162 15311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:31.540406 15311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:31.547605 15311 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:33193
I20260807 18:38:31.547644 15415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:33193 every 8 connection(s)
I20260807 18:38:31.548667 15311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:31.552096 15419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:31.557631 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 15311
I20260807 18:38:31.557786 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:31.558109 6697 external_mini_cluster.cc:1428] Setting key 2fe91960014393a21734d2fa7b8c2564
I20260807 18:38:31.558882 15419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2: Bootstrap starting.
I20260807 18:38:31.561724 15419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:31.562705 15419 log.cc:824] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:31.565261 15419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2: No bootstrap required, opened a new log
I20260807 18:38:31.568408 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:31.559661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.6.138.126:33193)
0807 18:38:31.560426 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:31.560473 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:31.560513 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:38:31.561162 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:31.561179 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:31.561608 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:31.561964 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:31.562339 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.563547 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.564887 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.565417 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.565558 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:31.566499 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:31.566504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:31.566517 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:31.566926 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:31.566991 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:31.567015 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:31.567094 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:31.567291 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 18:38:31.567613 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":141,"threads_started":1}
I20260807 18:38:31.569682 15419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ed0f5489542b3b6f54c3173f8bbb2" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33193 } }
I20260807 18:38:31.569938 15419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:31.570039 15419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 347ed0f5489542b3b6f54c3173f8bbb2, State: Initialized, Role: FOLLOWER
I20260807 18:38:31.570582 15419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ed0f5489542b3b6f54c3173f8bbb2" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33193 } }
I20260807 18:38:31.570788 15419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:31.570876 15419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:31.571034 15419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:31.571624 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:38:31.572229 15419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ed0f5489542b3b6f54c3173f8bbb2" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33193 } }
I20260807 18:38:31.572696 15419 leader_election.cc:302] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 347ed0f5489542b3b6f54c3173f8bbb2; no voters:
I20260807 18:38:31.573132 15419 leader_election.cc:288] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:31.573208 15428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:31.573436 15428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [term 1 LEADER]: Becoming Leader. State: Replica: 347ed0f5489542b3b6f54c3173f8bbb2, State: Running, Role: LEADER
I20260807 18:38:31.573776 15428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ed0f5489542b3b6f54c3173f8bbb2" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33193 } }
I20260807 18:38:31.574534 15419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:31.575053 15428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 347ed0f5489542b3b6f54c3173f8bbb2. Latest consensus state: current_term: 1 leader_uuid: "347ed0f5489542b3b6f54c3173f8bbb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ed0f5489542b3b6f54c3173f8bbb2" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33193 } } }
I20260807 18:38:31.575181 15428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:31.575676 15432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "347ed0f5489542b3b6f54c3173f8bbb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347ed0f5489542b3b6f54c3173f8bbb2" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33193 } } }
I20260807 18:38:31.575799 15432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:31.576107 15435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:31.579794 15435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:31.585578 15435 catalog_manager.cc:1347] Generated new cluster ID: ab118041d2f64508bba64629d81cbdc3
I20260807 18:38:31.585693 15435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:31.605031 15435 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:31.606261 15435 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:31.614770 15435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 347ed0f5489542b3b6f54c3173f8bbb2: Generated new TSK 0
I20260807 18:38:31.615533 15435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:38:31.638675 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:31.632873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44226 (local address 127.6.138.126:33193)
0807 18:38:31.633022 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:31.633026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:31.633209 (+ 183us) server_negotiation.cc:407] Connection header received
0807 18:38:31.633354 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:31.633356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:31.633406 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:31.633508 (+ 102us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:38:31.633806 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.634378 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.635807 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.636100 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.636193 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:31.636650 (+ 457us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:38:31.638277 (+ 1627us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43307/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:38:31.638519 (+ 242us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44226: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43307/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 18:38:31.638752 15426 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44226: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43307/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 18:38:31.640527 15463 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:31.632721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:33193 (local address 127.0.0.1:44226)
0807 18:38:31.633077 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:31.633095 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:31.633226 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:31.633542 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:31.633546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:31.633556 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:38:31.633689 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:31.633696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.634513 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.634518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:31.635641 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:31.635654 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.636271 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.636276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:31.636383 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:31.636418 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:38:31.638485 (+ 2067us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43307/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:38:31.640316 (+ 1831us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.138.126:33193: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43307/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":1564,"thread_start_us":125,"threads_started":1}
I20260807 18:38:31.641083 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 15311
E20260807 18:38:31.657716 15464 webserver.cc:533] Webserver: sslize failed:
E20260807 18:38:31.657830 15464 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T18:38:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (336 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T18:38:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:31Z Disabled control of system clock
I20260807 18:38:31.694970 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38855
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:38855
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/testchainca.cert with env {}
W20260807 18:38:31.809873 15469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:31.810148 15469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:31.810187 15469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:31.814417 15469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:31.814496 15469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:31.814513 15469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:31.814531 15469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:31.814546 15469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:38:31.814563 15469 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:38:31.814575 15469 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 18:38:31.820055 15469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:38855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:31.821271 15469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:31.822252 15469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:31.831298 15475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:31.831337 15477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:31.831298 15481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:31.831696 15469 server_base.cc:1034] running on GCE node
I20260807 18:38:31.832347 15469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:31.833614 15469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:31.834817 15469 hybrid_clock.cc:648] HybridClock initialized: now 1786127911834773 us; error 57 us; skew 500 ppm
I20260807 18:38:31.836846 15469 webserver.cc:460] Webserver started at http://127.6.138.126:34413/ using document root <none> and password file <none>
I20260807 18:38:31.837471 15469 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:31.837571 15469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:31.837818 15469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:31.839927 15469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "663d72c416694b07b56d1642b8075380"
format_stamp: "Formatted at 2026-08-07 18:38:31 on dist-test-slave-2d19"
server_key: "5e96a8a02fba84a1e6a94f84e9085872"
server_key_iv: "6de0801dfc620d0e5f94296919969ace"
server_key_version: "encryptionkey@0"
I20260807 18:38:31.840555 15469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "663d72c416694b07b56d1642b8075380"
format_stamp: "Formatted at 2026-08-07 18:38:31 on dist-test-slave-2d19"
server_key: "5e96a8a02fba84a1e6a94f84e9085872"
server_key_iv: "6de0801dfc620d0e5f94296919969ace"
server_key_version: "encryptionkey@0"
I20260807 18:38:31.844449 15469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:38:31.846985 15489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:31.848353 15469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:31.848515 15469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "663d72c416694b07b56d1642b8075380"
format_stamp: "Formatted at 2026-08-07 18:38:31 on dist-test-slave-2d19"
server_key: "5e96a8a02fba84a1e6a94f84e9085872"
server_key_iv: "6de0801dfc620d0e5f94296919969ace"
server_key_version: "encryptionkey@0"
I20260807 18:38:31.848650 15469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:31.867503 15469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:31.868513 15469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:31.868736 15469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:31.876710 15573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:38855 every 8 connection(s)
I20260807 18:38:31.876729 15469 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:38855
I20260807 18:38:31.877825 15469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:31.881070 15574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:31.881613 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 15469
I20260807 18:38:31.881753 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:31.882058 6697 external_mini_cluster.cc:1428] Setting key 74bc828a0590ae8bcc8365aec3227258
I20260807 18:38:31.887883 15574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380: Bootstrap starting.
I20260807 18:38:31.890755 15574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:31.891567 15574 log.cc:824] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:31.893078 15580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:31.883574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60868 (local address 127.6.138.126:38855)
0807 18:38:31.884345 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:31.884364 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:31.884411 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:38:31.885244 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:31.885268 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:31.885799 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:31.886178 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:31.886640 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.888091 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.889132 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.889835 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.890006 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:31.891050 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:31.891057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:31.891076 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:38:31.891487 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:31.891557 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:38:31.891579 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:31.891698 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:31.891882 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 18:38:31.892185 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":111,"threads_started":1}
I20260807 18:38:31.893891 15574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380: No bootstrap required, opened a new log
I20260807 18:38:31.896010 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:38:31.898070 15574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663d72c416694b07b56d1642b8075380" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38855 } }
I20260807 18:38:31.898331 15574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:31.898423 15574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663d72c416694b07b56d1642b8075380, State: Initialized, Role: FOLLOWER
I20260807 18:38:31.898860 15574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663d72c416694b07b56d1642b8075380" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38855 } }
I20260807 18:38:31.899024 15574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:31.899116 15574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:31.899243 15574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:31.900223 15574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663d72c416694b07b56d1642b8075380" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38855 } }
I20260807 18:38:31.900599 15574 leader_election.cc:302] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 663d72c416694b07b56d1642b8075380; no voters:
I20260807 18:38:31.900985 15574 leader_election.cc:288] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:31.901098 15583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:31.901358 15583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [term 1 LEADER]: Becoming Leader. State: Replica: 663d72c416694b07b56d1642b8075380, State: Running, Role: LEADER
I20260807 18:38:31.901734 15583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663d72c416694b07b56d1642b8075380" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38855 } }
I20260807 18:38:31.902320 15574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:31.903559 15585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "663d72c416694b07b56d1642b8075380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663d72c416694b07b56d1642b8075380" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38855 } } }
I20260807 18:38:31.903553 15586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 663d72c416694b07b56d1642b8075380. Latest consensus state: current_term: 1 leader_uuid: "663d72c416694b07b56d1642b8075380" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663d72c416694b07b56d1642b8075380" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38855 } } }
I20260807 18:38:31.903720 15585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:31.903720 15586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:31.904165 15595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:31.907358 15595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:31.913208 15595 catalog_manager.cc:1347] Generated new cluster ID: c9ca9be74c784621893a5f9be69ca6d5
I20260807 18:38:31.913280 15595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:31.938894 15595 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:31.940228 15595 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:31.960073 15595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 663d72c416694b07b56d1642b8075380: Generated new TSK 0
I20260807 18:38:31.961033 15595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:38:32.198179 15580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:31.970420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60882 (local address 127.6.138.126:38855)
0807 18:38:31.970561 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:31.970566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:31.970743 (+ 177us) server_negotiation.cc:407] Connection header received
0807 18:38:31.970949 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:31.970952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:31.971001 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:31.971118 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:38:31.971381 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.971954 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.972965 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.973203 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.973322 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:31.973674 (+ 352us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:38:32.197589 (+223915us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:38:32.197976 (+ 387us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60882: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 18:38:32.198271 15626 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:31.970308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38855 (local address 127.0.0.1:60882)
0807 18:38:31.970651 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:31.970664 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:31.970797 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:31.971131 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:31.971134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:31.971143 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:38:31.971257 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:31.971269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.972111 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.972115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:31.972819 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:31.972829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:31.973333 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:31.973337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:31.973427 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:31.973460 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:38:32.197870 (+224410us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:38:32.198086 (+ 216us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.138.126:38855: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":125,"threads_started":1}
W20260807 18:38:32.198297 15580 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60882: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260807 18:38:32.199126 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 15469
2026-08-07T18:38:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (542 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (133 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (134 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:32 dist-test-slave-2d19 krb5kdc[15646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:32 dist-test-slave-2d19 krb5kdc[15646](info): set up 2 sockets
Aug 07 18:38:32 dist-test-slave-2d19 krb5kdc[15646](info): commencing operation
krb5kdc: starting...
W20260807 18:38:33.737852 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:33 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:33.879987 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:35087
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:35087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:33.998438 15665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:33.998788 15665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:33.998873 15665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:34.003453 15665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:34.003561 15665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:34.003605 15665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:34.003670 15665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:34.003711 15665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:34.009907 15665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:35087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:35087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:34.011235 15665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:34.012456 15665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:34.021895 15673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:34.021919 15672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:34.021914 15675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:34.022119 15665 server_base.cc:1034] running on GCE node
I20260807 18:38:34.022794 15665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:34.024014 15665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:34.025182 15665 hybrid_clock.cc:648] HybridClock initialized: now 1786127914025179 us; error 61 us; skew 500 ppm
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.031824 15665 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:34.032955 15665 webserver.cc:460] Webserver started at http://127.6.138.126:42063/ using document root <none> and password file <none>
I20260807 18:38:34.033569 15665 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:34.033636 15665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:34.033893 15665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:34.036075 15665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "fd5bee7b820848feb11f7aa423f0a5cd"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "3a34feb5790208b40fad7f914951b3ee"
server_key_iv: "a5c9d88bf81ce9f64572d6fa0df14db7"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.036736 15665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "fd5bee7b820848feb11f7aa423f0a5cd"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "3a34feb5790208b40fad7f914951b3ee"
server_key_iv: "a5c9d88bf81ce9f64572d6fa0df14db7"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.040750 15665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:38:34.043295 15685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.044598 15665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:34.044750 15665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "fd5bee7b820848feb11f7aa423f0a5cd"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "3a34feb5790208b40fad7f914951b3ee"
server_key_iv: "a5c9d88bf81ce9f64572d6fa0df14db7"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.044867 15665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:34.056111 15665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:34.059458 15665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:34.059677 15665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:34.067066 15665 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:35087
I20260807 18:38:34.067085 15775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:35087 every 8 connection(s)
I20260807 18:38:34.068053 15665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:34.070873 15776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:34.076625 15776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd: Bootstrap starting.
I20260807 18:38:34.077699 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 15665
I20260807 18:38:34.077849 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:34.078172 6697 external_mini_cluster.cc:1428] Setting key 101ed49f5328229e258755bb637b99c4
I20260807 18:38:34.079613 15776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:34.080456 15776 log.cc:824] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:34.082401 15776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd: No bootstrap required, opened a new log
I20260807 18:38:34.085927 15776 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5bee7b820848feb11f7aa423f0a5cd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35087 } }
I20260807 18:38:34.086150 15776 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:34.086226 15776 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd5bee7b820848feb11f7aa423f0a5cd, State: Initialized, Role: FOLLOWER
I20260807 18:38:34.086717 15776 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5bee7b820848feb11f7aa423f0a5cd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35087 } }
I20260807 18:38:34.086881 15776 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:34.086978 15776 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:34.087105 15776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:34.088343 15776 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5bee7b820848feb11f7aa423f0a5cd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35087 } }
I20260807 18:38:34.088752 15776 leader_election.cc:302] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd5bee7b820848feb11f7aa423f0a5cd; no voters:
I20260807 18:38:34.089183 15776 leader_election.cc:288] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:34.089334 15783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:34.089594 15783 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [term 1 LEADER]: Becoming Leader. State: Replica: fd5bee7b820848feb11f7aa423f0a5cd, State: Running, Role: LEADER
I20260807 18:38:34.089937 15783 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5bee7b820848feb11f7aa423f0a5cd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35087 } }
I20260807 18:38:34.090620 15776 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:34.091241 15783 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd5bee7b820848feb11f7aa423f0a5cd. Latest consensus state: current_term: 1 leader_uuid: "fd5bee7b820848feb11f7aa423f0a5cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5bee7b820848feb11f7aa423f0a5cd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35087 } } }
I20260807 18:38:34.091413 15783 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:34.091691 15788 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd5bee7b820848feb11f7aa423f0a5cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5bee7b820848feb11f7aa423f0a5cd" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35087 } } }
I20260807 18:38:34.091786 15788 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:34.091936 15799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:34.094815 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.079404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35000 (local address 127.6.138.126:35087)
0807 18:38:34.079884 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.079900 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.079935 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:38:34.080680 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.080698 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.081119 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.081500 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.082322 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.083349 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.084242 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.084769 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.084970 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.087720 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:34.087727 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:34.087745 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:34.087778 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:34.089896 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.090417 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.090424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.090429 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.090577 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.090912 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.090915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.090917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.091532 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:34.091732 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:34.093476 (+ 1744us) server_negotiation.cc:299] Negotiation successful
0807 18:38:34.093810 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":150,"threads_started":1}
I20260807 18:38:34.095561 15799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:34.101478 15799 catalog_manager.cc:1347] Generated new cluster ID: 50b01960a7d645c0bbffd4ccb8e69485
I20260807 18:38:34.101581 15799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:34.115141 15799 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:34.116056 15799 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:34.119539 15799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd5bee7b820848feb11f7aa423f0a5cd: Generated new TSK 0
I20260807 18:38:34.120204 15799 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:34.156764 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35087
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:34.277175 15817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:34.277441 15817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:34.277519 15817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:34.281507 15817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:34.281558 15817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:34.281652 15817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:34.286984 15817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:34.288080 15817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:34.289149 15817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:34.299798 15828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:34.299837 15832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:34.299804 15829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:34.300060 15817 server_base.cc:1034] running on GCE node
I20260807 18:38:34.300572 15817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:34.301258 15817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:34.302467 15817 hybrid_clock.cc:648] HybridClock initialized: now 1786127914302445 us; error 31 us; skew 500 ppm
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.305681 15817 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:34.306558 15817 webserver.cc:460] Webserver started at http://127.6.138.65:36845/ using document root <none> and password file <none>
I20260807 18:38:34.307114 15817 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:34.307171 15817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:34.307343 15817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:34.309479 15817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "91a82d38035147afab9aaf0068d2c9fb"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "7b72a85b1b3acbc414bfbb538276c898"
server_key_iv: "95e01018bb76045513c1ee13911616a6"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.310053 15817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "91a82d38035147afab9aaf0068d2c9fb"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "7b72a85b1b3acbc414bfbb538276c898"
server_key_iv: "95e01018bb76045513c1ee13911616a6"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.313814 15817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:34.316375 15842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.317662 15817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:34.317757 15817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "91a82d38035147afab9aaf0068d2c9fb"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "7b72a85b1b3acbc414bfbb538276c898"
server_key_iv: "95e01018bb76045513c1ee13911616a6"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.317842 15817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:34.331558 15817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:34.334972 15817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:34.335129 15817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:34.335678 15817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:34.336500 15817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:34.336548 15817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.336587 15817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:34.336606 15817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.345861 15817 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:33115
I20260807 18:38:34.345885 16020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:33115 every 8 connection(s)
I20260807 18:38:34.346889 15817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:34.355073 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 15817
I20260807 18:38:34.355232 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:34.355556 6697 external_mini_cluster.cc:1428] Setting key 515882713110e1ee3e959179a85ce2b2
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:34.362749 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.349024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:41111 (local address 127.6.138.126:35087)
0807 18:38:34.349198 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.349204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.350177 (+ 973us) server_negotiation.cc:407] Connection header received
0807 18:38:34.351570 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.351573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.351623 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.351703 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.353074 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.353618 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.354813 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.355164 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.355282 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.358426 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:34.358433 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:34.358439 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:34.358474 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:34.360173 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.360875 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.360877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.360878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.360960 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.361393 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.361396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.361397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.361595 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:34.361676 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:34.362353 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 18:38:34.362514 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:38:34.363555 16023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.349455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:35087 (local address 127.6.138.65:41111)
0807 18:38:34.350010 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:34.350058 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:34.351348 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:34.351886 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:34.351894 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:34.352406 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:34.352919 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:34.352934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.353736 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.353738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:34.354670 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:34.354680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.355365 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.355369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:34.355533 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.356314 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:34.356334 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:34.358242 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:34.360327 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:34.360338 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:34.360358 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:34.360769 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:34.361066 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:34.361071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:34.361075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:34.361287 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:34.361723 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:34.361728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:34.362055 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 18:38:34.362229 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 18:38:34.362518 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":111,"threads_started":1}
I20260807 18:38:34.365165 16021 heartbeater.cc:347] Connected to a master server at 127.6.138.126:35087
I20260807 18:38:34.365442 16021 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:34.366037 16021 heartbeater.cc:511] Master 127.6.138.126:35087 requested a full tablet report, sending...
I20260807 18:38:34.367690 15713 ts_manager.cc:194] Registered new tserver with Master: 91a82d38035147afab9aaf0068d2c9fb (127.6.138.65:33115)
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:38:34.368934 15713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:41111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:34.391760 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35087
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:34.509553 16029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:34.509807 16029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:34.509855 16029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:34.513939 16029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:34.514003 16029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:34.514061 16029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:34.519382 16029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:34.520565 16029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:34.521723 16029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:38:34.531667 16029 server_base.cc:1034] running on GCE node
W20260807 18:38:34.531649 16040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:34.531652 16037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:34.531661 16038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:34.532508 16029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:34.533139 16029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:34.534332 16029 hybrid_clock.cc:648] HybridClock initialized: now 1786127914534299 us; error 41 us; skew 500 ppm
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.537231 16029 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:34.538151 16029 webserver.cc:460] Webserver started at http://127.6.138.66:38899/ using document root <none> and password file <none>
I20260807 18:38:34.538733 16029 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:34.538828 16029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:34.539109 16029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:34.541311 16029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "1d5cb0f7a38d4a8eb39037a29d90268c"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "9895be1bdea6e56c660c1d73b933e78d"
server_key_iv: "392a5afe3a1399f414ffbdcd86eb73ec"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.541890 16029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "1d5cb0f7a38d4a8eb39037a29d90268c"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "9895be1bdea6e56c660c1d73b933e78d"
server_key_iv: "392a5afe3a1399f414ffbdcd86eb73ec"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.545863 16029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:34.548382 16053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.549952 16029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:34.550107 16029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "1d5cb0f7a38d4a8eb39037a29d90268c"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "9895be1bdea6e56c660c1d73b933e78d"
server_key_iv: "392a5afe3a1399f414ffbdcd86eb73ec"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.550230 16029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:34.560715 16029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:34.563949 16029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:34.564177 16029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:34.564822 16029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:34.565706 16029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:34.565783 16029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.565852 16029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:34.565904 16029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.575197 16029 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:44473
I20260807 18:38:34.575228 16238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:44473 every 8 connection(s)
I20260807 18:38:34.576115 16029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:34.579871 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 16029
I20260807 18:38:34.580009 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:34.580404 6697 external_mini_cluster.cc:1428] Setting key b2bf9431f48ccf464c2637599319cda7
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:34.591439 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.578045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:40705 (local address 127.6.138.126:35087)
0807 18:38:34.578229 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.578234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.579112 (+ 878us) server_negotiation.cc:407] Connection header received
0807 18:38:34.580719 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.580723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.580787 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.580880 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.582447 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.583285 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.584607 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.584834 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.584952 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.587463 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:34.587467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:34.587469 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:34.587502 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:34.589114 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.589749 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.589751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.589752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.589828 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.590252 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.590255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.590256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.590428 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:34.590569 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:34.591132 (+ 563us) server_negotiation.cc:299] Negotiation successful
0807 18:38:34.591260 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:38:34.592535 16243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.578439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:35087 (local address 127.6.138.66:40705)
0807 18:38:34.578944 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:34.578987 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:34.580436 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:34.581106 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:34.581113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:34.581672 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:34.582293 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:34.582308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.583422 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.583425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:34.584450 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:34.584460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.584997 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.585001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:34.585175 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.585806 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:34.585822 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:34.587284 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:34.589247 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:34.589256 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:34.589274 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:34.589645 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:34.589927 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:34.589933 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:34.589938 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:34.590152 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:34.590604 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:34.590611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:34.590863 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 18:38:34.591069 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 18:38:34.591413 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":127,"threads_started":1}
I20260807 18:38:34.593761 16240 heartbeater.cc:347] Connected to a master server at 127.6.138.126:35087
I20260807 18:38:34.594018 16240 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:34.594607 16240 heartbeater.cc:511] Master 127.6.138.126:35087 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
I20260807 18:38:34.595919 15713 ts_manager.cc:194] Registered new tserver with Master: 1d5cb0f7a38d4a8eb39037a29d90268c (127.6.138.66:44473)
I20260807 18:38:34.596868 15713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:40705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:34.613375 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35087
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:34.729053 16246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:34.729300 16246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:34.729346 16246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:34.733433 16246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:34.733493 16246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:34.733552 16246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:34.738852 16246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:34.740059 16246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:34.741174 16246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:34.751407 16251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:34.751407 16256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:34.751498 16254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:34.751837 16246 server_base.cc:1034] running on GCE node
I20260807 18:38:34.752394 16246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:34.753106 16246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:34.754380 16246 hybrid_clock.cc:648] HybridClock initialized: now 1786127914754359 us; error 31 us; skew 500 ppm
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.757464 16246 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:34.758419 16246 webserver.cc:460] Webserver started at http://127.6.138.67:33405/ using document root <none> and password file <none>
I20260807 18:38:34.759021 16246 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:34.759088 16246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:34.759316 16246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:34.761580 16246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "9e8291315931475abf94367b3d16a4e3"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "d72358ef22127ec01b8d4147d0e9253c"
server_key_iv: "c7ee822980328f70335a8ea7fc5229c7"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.762169 16246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "9e8291315931475abf94367b3d16a4e3"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "d72358ef22127ec01b8d4147d0e9253c"
server_key_iv: "c7ee822980328f70335a8ea7fc5229c7"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.766144 16246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:34.768747 16266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.770144 16246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:34.770246 16246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "9e8291315931475abf94367b3d16a4e3"
format_stamp: "Formatted at 2026-08-07 18:38:34 on dist-test-slave-2d19"
server_key: "d72358ef22127ec01b8d4147d0e9253c"
server_key_iv: "c7ee822980328f70335a8ea7fc5229c7"
server_key_version: "encryptionkey@0"
I20260807 18:38:34.770387 16246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:34.782605 16246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:34.786095 16246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:34.786314 16246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:34.786976 16246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:34.787842 16246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:34.787891 16246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.787967 16246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:34.788002 16246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:34.797845 16246 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:39277
I20260807 18:38:34.797891 16445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:39277 every 8 connection(s)
I20260807 18:38:34.798872 16246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:34.801443 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 16246
I20260807 18:38:34.801594 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:34.801898 6697 external_mini_cluster.cc:1428] Setting key fd0972c5083854ea31a76b6dfac30f16
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:34.813704 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.800841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:50627 (local address 127.6.138.126:35087)
0807 18:38:34.800984 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.800988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.802019 (+ 1031us) server_negotiation.cc:407] Connection header received
0807 18:38:34.803605 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.803608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.803653 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.803721 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.805046 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.805576 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.806774 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.806988 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.807076 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.809741 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:34.809744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:34.809756 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:34.809783 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:34.811319 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.811849 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.811852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.811853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.811953 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.812519 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.812522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.812523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.812731 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:34.812819 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:34.813443 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 18:38:34.813575 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:38:34.814599 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.801279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:35087 (local address 127.6.138.67:50627)
0807 18:38:34.801848 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:34.801902 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:34.803359 (+ 1457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:34.803903 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:34.803913 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:34.804394 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:34.804886 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:34.804901 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.805701 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.805704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:34.806650 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:34.806661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.807153 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.807157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:34.807307 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.808047 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:34.808065 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:34.809574 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:34.811447 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:34.811452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:34.811465 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:34.811745 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:34.812110 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:34.812114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:34.812117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:34.812402 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:34.812866 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:34.812872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:34.813149 (+ 277us) client_negotiation.cc:769] Sending connection context
0807 18:38:34.813329 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 18:38:34.813656 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":138,"threads_started":1}
I20260807 18:38:34.815913 16447 heartbeater.cc:347] Connected to a master server at 127.6.138.126:35087
I20260807 18:38:34.816183 16447 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:34.816728 16447 heartbeater.cc:511] Master 127.6.138.126:35087 requested a full tablet report, sending...
I20260807 18:38:34.817817 15713 ts_manager.cc:194] Registered new tserver with Master: 9e8291315931475abf94367b3d16a4e3 (127.6.138.67:39277)
I20260807 18:38:34.818346 15713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:50627
I20260807 18:38:34.828301 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/test-admin.keytab.
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.849491 6697 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 18:38:34.852859 16456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:34.851040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56684 (local address 127.6.138.65:33115)
0807 18:38:34.851450 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.851469 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.851496 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:38:34.851551 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.851556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.851753 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.851900 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.852627 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56684: BlockingRecv error: recv got EOF from 127.0.0.1:56684 (error 108)
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":109,"threads_started":1}
W20260807 18:38:34.852941 6697 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.854308 6697 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.855396 6697 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:34.865397 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.856412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.6.138.65:33115)
0807 18:38:34.856598 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.856601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.856632 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:38:34.856758 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.856761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.856802 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.856871 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.857433 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.858226 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.859058 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.859472 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.859558 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.861970 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:34.861979 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:34.861994 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:34.862017 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:34.863524 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.863940 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.863945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.863946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.864041 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.864375 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.864380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.864381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.864665 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:34.864909 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:34.865093 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 18:38:34.865252 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:38:34.869486 16456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:34.868347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56700 (local address 127.6.138.65:33115)
0807 18:38:34.868498 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.868504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.868596 (+ 92us) server_negotiation.cc:407] Connection header received
0807 18:38:34.868760 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.868763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.868807 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.868917 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.869360 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56700: BlockingRecv error: recv got EOF from 127.0.0.1:56700 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 18:38:34.869617 6697 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.870639 6697 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.871505 6697 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:34.880954 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.872376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56702 (local address 127.6.138.65:33115)
0807 18:38:34.872509 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.872514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.872631 (+ 117us) server_negotiation.cc:407] Connection header received
0807 18:38:34.872773 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.872776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.872820 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.872904 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.873419 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.873942 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.874759 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.874988 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.875103 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.877280 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:34.877284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:34.877287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:34.877315 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:34.878924 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.879402 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.879405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.879407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.879509 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.879863 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.879869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.879876 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.880132 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:34.880298 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:34.880625 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:38:34.880758 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 18:38:34.883734 16456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:34.882855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.6.138.65:33115)
0807 18:38:34.883006 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.883010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.883037 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:38:34.883149 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.883151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.883185 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.883249 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.883607 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56706: BlockingRecv error: recv got EOF from 127.0.0.1:56706 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 18:38:34.884052 6697 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127881318083-6697-0/krb5kdc/krb5cc)
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.885193 6697 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:34.886134 6697 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:38:34 dist-test-slave-2d19 krb5kdc[15646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:38:34.895613 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:34.887011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56708 (local address 127.6.138.65:33115)
0807 18:38:34.887158 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:34.887161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:34.887191 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:38:34.887286 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:34.887287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:34.887322 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:34.887381 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:34.887827 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.888433 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.889211 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:34.889419 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:34.889500 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:34.891929 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:34.891934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:34.891937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:34.891990 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:34.893602 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.894094 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.894098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.894099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.894189 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:34.894536 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:34.894539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:34.894540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:34.894737 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:34.894854 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:34.895233 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 18:38:34.895432 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:38:34.896770 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 15817
I20260807 18:38:34.910323 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 16029
I20260807 18:38:34.934233 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 16246
I20260807 18:38:34.958000 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 15665
2026-08-07T18:38:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2492 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:35 dist-test-slave-2d19 krb5kdc[16481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:35 dist-test-slave-2d19 krb5kdc[16481](info): set up 2 sockets
Aug 07 18:38:35 dist-test-slave-2d19 krb5kdc[16481](info): commencing operation
krb5kdc: starting...
W20260807 18:38:36.237991 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:36 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.126.keytab.
Entry for principal oryx/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.126.keytab.
I20260807 18:38:36.389783 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38925
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.126.keytab
--principal=oryx/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:38925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:36.511405 16500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:36.511788 16500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:36.511878 16500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:36.516098 16500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:36.516227 16500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:36.516268 16500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:36.516319 16500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:36.516353 16500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:36.521907 16500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:38925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.126.keytab
--principal=oryx/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:36.523131 16500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:36.524271 16500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:36.533631 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:36.533663 16514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:36.533681 16508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:36.534042 16500 server_base.cc:1034] running on GCE node
I20260807 18:38:36.534665 16500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:36.535835 16500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:36.537006 16500 hybrid_clock.cc:648] HybridClock initialized: now 1786127916536979 us; error 42 us; skew 500 ppm
Aug 07 18:38:36 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127916, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:36.540012 16500 init.cc:377] Logged in from keytab as oryx/127.6.138.126@KRBTEST.COM (short username oryx)
I20260807 18:38:36.541146 16500 webserver.cc:460] Webserver started at http://127.6.138.126:35403/ using document root <none> and password file <none>
I20260807 18:38:36.541813 16500 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:36.541923 16500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:36.542192 16500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:36.544474 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "aecb998e302b4c218cfdbe152b4da9b1"
format_stamp: "Formatted at 2026-08-07 18:38:36 on dist-test-slave-2d19"
server_key: "88d6144f158a601363aa510d28ab8e8a"
server_key_iv: "4f918fb7ce9b5d0e3dda99303794ee7a"
server_key_version: "encryptionkey@0"
I20260807 18:38:36.545150 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "aecb998e302b4c218cfdbe152b4da9b1"
format_stamp: "Formatted at 2026-08-07 18:38:36 on dist-test-slave-2d19"
server_key: "88d6144f158a601363aa510d28ab8e8a"
server_key_iv: "4f918fb7ce9b5d0e3dda99303794ee7a"
server_key_version: "encryptionkey@0"
I20260807 18:38:36.549319 16500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:38:36.552162 16527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:36.553737 16500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:36.553925 16500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "aecb998e302b4c218cfdbe152b4da9b1"
format_stamp: "Formatted at 2026-08-07 18:38:36 on dist-test-slave-2d19"
server_key: "88d6144f158a601363aa510d28ab8e8a"
server_key_iv: "4f918fb7ce9b5d0e3dda99303794ee7a"
server_key_version: "encryptionkey@0"
I20260807 18:38:36.554075 16500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:36.574435 16500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:36.577934 16500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:36.578173 16500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:36.585702 16500 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:38925
I20260807 18:38:36.586018 16611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:38925 every 8 connection(s)
I20260807 18:38:36.586786 16500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:36.588531 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 16500
I20260807 18:38:36.588682 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:36.588994 6697 external_mini_cluster.cc:1428] Setting key a2fc3e653fa04a3949807b270281a4a0
I20260807 18:38:36.590175 16613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:36.596915 16613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: Bootstrap starting.
Aug 07 18:38:36 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:36.599623 16613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:36.600471 16613 log.cc:824] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:36.602854 16613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: No bootstrap required, opened a new log
I20260807 18:38:36.604769 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:36.590433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.6.138.126:38925)
0807 18:38:36.590853 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:36.590870 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:36.590906 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:38:36.591555 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:36.591570 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:36.591901 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:36.592178 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:36.593119 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:36.594138 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:36.595273 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:36.595765 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:36.595883 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:36.599176 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:36.599182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:36.599197 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:36.599229 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:36.601614 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:36.602122 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:36.602127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:36.602132 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:36.602249 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:36.602552 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:36.602554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:36.602555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:36.603021 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:36.603195 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:36.603509 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 18:38:36.603852 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":136,"threads_started":1}
I20260807 18:38:36.606686 16613 raft_consensus.cc:348] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } }
I20260807 18:38:36.606946 16613 raft_consensus.cc:374] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:36.607038 16613 raft_consensus.cc:729] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aecb998e302b4c218cfdbe152b4da9b1, State: Initialized, Role: FOLLOWER
I20260807 18:38:36.607543 16613 consensus_queue.cc:260] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } }
I20260807 18:38:36.607704 16613 raft_consensus.cc:388] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:36.607775 16613 raft_consensus.cc:482] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:36.607875 16613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:36.608938 16613 raft_consensus.cc:504] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } }
I20260807 18:38:36.609292 16613 leader_election.cc:302] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aecb998e302b4c218cfdbe152b4da9b1; no voters:
I20260807 18:38:36.609655 16613 leader_election.cc:288] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:36.609741 16623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:36.609987 16623 raft_consensus.cc:686] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 1 LEADER]: Becoming Leader. State: Replica: aecb998e302b4c218cfdbe152b4da9b1, State: Running, Role: LEADER
I20260807 18:38:36.610352 16623 consensus_queue.cc:237] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } }
I20260807 18:38:36.611016 16613 sys_catalog.cc:564] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:36.612308 16624 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } }
I20260807 18:38:36.612461 16624 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:36.612783 16625 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aecb998e302b4c218cfdbe152b4da9b1. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } }
I20260807 18:38:36.612871 16625 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:36.612903 16638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:36.616298 16638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:36.622735 16638 catalog_manager.cc:1347] Generated new cluster ID: 589dd4e6265641c3bb2a26ad1e6a6187
I20260807 18:38:36.622817 16638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:36.652998 16638 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:36.654312 16638 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:36.671306 16638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: Generated new TSK 0
I20260807 18:38:36.672317 16638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab.
Entry for principal oryx/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab.
I20260807 18:38:36.716215 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38925
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab
--principal=oryx/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:36.837206 16656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:36.837477 16656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:36.837524 16656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:36.841684 16656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:36.841749 16656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:36.841835 16656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:36.847410 16656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab
--principal=oryx/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:36.848665 16656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:36.849735 16656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:36.860270 16667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:36.860234 16665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:36.860561 16656 server_base.cc:1034] running on GCE node
W20260807 18:38:36.860234 16663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:36.861081 16656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:36.861765 16656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:36.862969 16656 hybrid_clock.cc:648] HybridClock initialized: now 1786127916862932 us; error 52 us; skew 500 ppm
Aug 07 18:38:36 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127916, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:36.866007 16656 init.cc:377] Logged in from keytab as oryx/127.6.138.65@KRBTEST.COM (short username oryx)
I20260807 18:38:36.866986 16656 webserver.cc:460] Webserver started at http://127.6.138.65:37763/ using document root <none> and password file <none>
I20260807 18:38:36.867597 16656 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:36.867697 16656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:36.867951 16656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:36.870229 16656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "52badaa1b1204ab1b901c2d11e7e1a79"
format_stamp: "Formatted at 2026-08-07 18:38:36 on dist-test-slave-2d19"
server_key: "5b4fa31a80d29560681574e7aee21e3c"
server_key_iv: "39d3903ea9f2353ee3babf8621dcaf03"
server_key_version: "encryptionkey@0"
I20260807 18:38:36.870862 16656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "52badaa1b1204ab1b901c2d11e7e1a79"
format_stamp: "Formatted at 2026-08-07 18:38:36 on dist-test-slave-2d19"
server_key: "5b4fa31a80d29560681574e7aee21e3c"
server_key_iv: "39d3903ea9f2353ee3babf8621dcaf03"
server_key_version: "encryptionkey@0"
I20260807 18:38:36.874969 16656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:38:36.877816 16682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:36.879209 16656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:38:36.879360 16656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "52badaa1b1204ab1b901c2d11e7e1a79"
format_stamp: "Formatted at 2026-08-07 18:38:36 on dist-test-slave-2d19"
server_key: "5b4fa31a80d29560681574e7aee21e3c"
server_key_iv: "39d3903ea9f2353ee3babf8621dcaf03"
server_key_version: "encryptionkey@0"
I20260807 18:38:36.879487 16656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:36.922541 16656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:36.925992 16656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:36.926230 16656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:36.927228 16656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:36.928309 16656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:36.928398 16656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:36.928476 16656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:36.928541 16656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:36.939484 16656 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:43745
I20260807 18:38:36.939514 16854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:43745 every 8 connection(s)
I20260807 18:38:36.940640 16656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:36.945353 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 16656
I20260807 18:38:36.945498 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:36.945859 6697 external_mini_cluster.cc:1428] Setting key 71658930aaf8bf4a423f5ecd84c83416
Aug 07 18:38:36 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127916, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:38:36.958038 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:36.942703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:32895 (local address 127.6.138.126:38925)
0807 18:38:36.942874 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:36.942879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:36.943824 (+ 945us) server_negotiation.cc:407] Connection header received
0807 18:38:36.945214 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:36.945216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:36.945265 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:36.945342 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:36.947088 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:36.947787 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:36.949729 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:36.950194 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:36.950324 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:36.953557 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:36.953564 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:36.953570 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:36.953612 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:36.955460 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:36.956093 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:36.956099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:36.956103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:36.956210 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:36.956661 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:36.956667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:36.956669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:36.956940 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:36.957075 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:36.957638 (+ 563us) server_negotiation.cc:299] Negotiation successful
0807 18:38:36.957794 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:38:36.958956 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:36.943147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.65:32895)
0807 18:38:36.943650 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:36.943695 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:36.944929 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:36.945567 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:36.945577 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:36.946202 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:36.946886 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:36.946904 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:36.947950 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:36.947957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:36.949545 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:36.949573 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:36.950581 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:36.950586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:36.950770 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:36.951496 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:36.951514 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:36.953297 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:36.955629 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:36.955635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:36.955652 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:36.955944 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:36.956344 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:36.956347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:36.956349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:36.956511 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:36.957066 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:36.957074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:36.957352 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 18:38:36.957526 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 18:38:36.957921 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":123,"threads_started":1}
I20260807 18:38:36.960628 16857 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38925
I20260807 18:38:36.960908 16857 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.66@KRBTEST.COM" created.
I20260807 18:38:36.961581 16857 heartbeater.cc:511] Master 127.6.138.126:38925 requested a full tablet report, sending...
I20260807 18:38:36.963485 16554 ts_manager.cc:194] Registered new tserver with Master: 52badaa1b1204ab1b901c2d11e7e1a79 (127.6.138.65:43745)
I20260807 18:38:36.964886 16554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:32895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab.
Entry for principal oryx/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab.
I20260807 18:38:36.979771 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38925
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab
--principal=oryx/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:37.100590 16866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:37.100872 16866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:37.100919 16866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:37.105276 16866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:37.105343 16866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:37.105439 16866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:37.111028 16866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab
--principal=oryx/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:37.112507 16866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:37.113684 16866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:37.124392 16874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.124384 16872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.124379 16877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:37.124583 16866 server_base.cc:1034] running on GCE node
I20260807 18:38:37.125085 16866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:37.125795 16866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:37.126968 16866 hybrid_clock.cc:648] HybridClock initialized: now 1786127917126930 us; error 51 us; skew 500 ppm
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:37.129997 16866 init.cc:377] Logged in from keytab as oryx/127.6.138.66@KRBTEST.COM (short username oryx)
I20260807 18:38:37.130975 16866 webserver.cc:460] Webserver started at http://127.6.138.66:43833/ using document root <none> and password file <none>
I20260807 18:38:37.131582 16866 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:37.131685 16866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:37.131946 16866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:37.134137 16866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "59751f6a7203483c9bdb4c47feda63ab"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "8bcc67eddb9e1ff7c9a9468c2e7fa2b9"
server_key_iv: "4edaecf47d130941de96014e48aa8ee5"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.134742 16866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "59751f6a7203483c9bdb4c47feda63ab"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "8bcc67eddb9e1ff7c9a9468c2e7fa2b9"
server_key_iv: "4edaecf47d130941de96014e48aa8ee5"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.138867 16866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 18:38:37.141527 16891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.142835 16866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:37.142988 16866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "59751f6a7203483c9bdb4c47feda63ab"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "8bcc67eddb9e1ff7c9a9468c2e7fa2b9"
server_key_iv: "4edaecf47d130941de96014e48aa8ee5"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.143129 16866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:37.157858 16866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:37.161470 16866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:37.161754 16866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:37.162432 16866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:37.163349 16866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:37.163426 16866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.163497 16866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:37.163546 16866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.173640 16866 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:44067
I20260807 18:38:37.173672 17073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:44067 every 8 connection(s)
I20260807 18:38:37.174680 16866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:37.178439 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 16866
I20260807 18:38:37.178654 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:37.179067 6697 external_mini_cluster.cc:1428] Setting key a1e64dc7f1b435dde3836ca604558893
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:38:37.193181 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.176894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:55775 (local address 127.6.138.126:38925)
0807 18:38:37.177076 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:37.177082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:37.178111 (+ 1029us) server_negotiation.cc:407] Connection header received
0807 18:38:37.180096 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:37.180100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:37.180172 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:37.180418 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:37.182239 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.183074 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.184623 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.185092 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.185244 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.188310 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:37.188317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:37.188324 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:37.188371 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:37.190502 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.191123 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.191127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.191130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.191236 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.191696 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.191703 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.191707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.192046 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:37.192179 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:37.192908 (+ 729us) server_negotiation.cc:299] Negotiation successful
0807 18:38:37.193043 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:38:37.194243 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.177340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.66:55775)
0807 18:38:37.177910 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:37.177974 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:37.179819 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:37.180647 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:37.180656 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:37.181212 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:37.182007 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.182032 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.183244 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.183250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.184466 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.184485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.185303 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.185307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.185509 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.186269 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:37.186287 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:37.188007 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:37.190661 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.190666 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.190683 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.190989 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.191361 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.191363 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.191364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.191516 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.192199 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:37.192205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:37.192615 (+ 410us) client_negotiation.cc:769] Sending connection context
0807 18:38:37.192811 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 18:38:37.193175 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":133,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.67@KRBTEST.COM" created.
I20260807 18:38:37.195840 17074 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38925
I20260807 18:38:37.196138 17074 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:37.196733 17074 heartbeater.cc:511] Master 127.6.138.126:38925 requested a full tablet report, sending...
I20260807 18:38:37.198043 16554 ts_manager.cc:194] Registered new tserver with Master: 59751f6a7203483c9bdb4c47feda63ab (127.6.138.66:44067)
I20260807 18:38:37.198671 16554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:55775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab.
Entry for principal oryx/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab.
I20260807 18:38:37.216481 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38925
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab
--principal=oryx/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:37.341148 17079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:37.341447 17079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:37.341493 17079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:37.345727 17079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:37.345794 17079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:37.345857 17079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:37.351691 17079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab
--principal=oryx/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:37.352917 17079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:37.354414 17079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:37.366365 17087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.366384 17091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.366358 17085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:37.366719 17079 server_base.cc:1034] running on GCE node
I20260807 18:38:37.367231 17079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:37.368057 17079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:37.369244 17079 hybrid_clock.cc:648] HybridClock initialized: now 1786127917369212 us; error 46 us; skew 500 ppm
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:37.372835 17079 init.cc:377] Logged in from keytab as oryx/127.6.138.67@KRBTEST.COM (short username oryx)
I20260807 18:38:37.374064 17079 webserver.cc:460] Webserver started at http://127.6.138.67:41225/ using document root <none> and password file <none>
I20260807 18:38:37.374774 17079 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:37.374894 17079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:37.375191 17079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:37.377822 17079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "4642c706185948dc8d65e6709d4ddff4"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "811db10c34da0f6c95a94c9c2d337afe"
server_key_iv: "e0b5f3760ba6f130f75842d1fe0404c2"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.378520 17079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "4642c706185948dc8d65e6709d4ddff4"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "811db10c34da0f6c95a94c9c2d337afe"
server_key_iv: "e0b5f3760ba6f130f75842d1fe0404c2"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.383185 17079 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:38:37.386281 17100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.387868 17079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:37.388114 17079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "4642c706185948dc8d65e6709d4ddff4"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "811db10c34da0f6c95a94c9c2d337afe"
server_key_iv: "e0b5f3760ba6f130f75842d1fe0404c2"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.388263 17079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:37.403541 17079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:37.407114 17079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:37.407361 17079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:37.408046 17079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:37.409024 17079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:37.409102 17079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.409180 17079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:37.409227 17079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.419430 17079 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:43671
I20260807 18:38:37.419514 17285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:43671 every 8 connection(s)
I20260807 18:38:37.420517 17079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:37.425312 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 17079
I20260807 18:38:37.425455 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:37.425801 6697 external_mini_cluster.cc:1428] Setting key ab379b261ef02546bf8366b6071950d4
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:37.436672 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.422617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:47349 (local address 127.6.138.126:38925)
0807 18:38:37.422810 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:37.422816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:37.423779 (+ 963us) server_negotiation.cc:407] Connection header received
0807 18:38:37.425312 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:37.425319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:37.425429 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:37.425542 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:37.427302 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.428049 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.429423 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.429654 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.429782 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.432658 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:37.432661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:37.432663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:37.432694 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:37.434215 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.434832 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.434835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.434837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.434986 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.435415 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.435418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.435419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.435623 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:37.435716 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:37.436410 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 18:38:37.436542 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:38:37.437609 17290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.423095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.67:47349)
0807 18:38:37.423602 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:37.423649 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:37.425045 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:37.425960 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:37.425968 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:37.426466 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:37.427135 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.427153 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.428242 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.428246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.429260 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.429271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.429839 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.429843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.430040 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.430718 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:37.430735 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:37.432451 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:37.434346 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.434352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.434366 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.434687 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.435089 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.435092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.435093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.435266 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.435769 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:37.435775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:37.436047 (+ 272us) client_negotiation.cc:769] Sending connection context
0807 18:38:37.436228 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 18:38:37.436610 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":122,"threads_started":1}
I20260807 18:38:37.439136 17288 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38925
I20260807 18:38:37.439462 17288 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:37.440016 17288 heartbeater.cc:511] Master 127.6.138.126:38925 requested a full tablet report, sending...
I20260807 18:38:37.441138 16554 ts_manager.cc:194] Registered new tserver with Master: 4642c706185948dc8d65e6709d4ddff4 (127.6.138.67:43671)
I20260807 18:38:37.441686 16554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:47349
I20260807 18:38:37.453079 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.138.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.125.keytab.
Entry for principal oryx/127.6.138.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.125.keytab.
I20260807 18:38:37.489681 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.125:35473
--webserver_interface=127.6.138.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.125.keytab
--principal=oryx/127.6.138.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:38925,127.6.138.125:35473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:37.616575 17293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:37.616899 17293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:37.616946 17293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:37.621417 17293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:37.621495 17293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:37.621515 17293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:37.621531 17293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:37.621547 17293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:37.627049 17293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:38925,127.6.138.125:35473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.125.keytab
--principal=oryx/127.6.138.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.125:35473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.138.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
W20260807 18:38:37.627287 17293 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.138.126:38925,127.6.138.125:35473'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 18:38:37.628388 17293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:37.629487 17293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:37.638726 17303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.638726 17307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.638731 17304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:37.639065 17293 server_base.cc:1034] running on GCE node
I20260807 18:38:37.640070 17293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:37.641396 17293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:37.642617 17293 hybrid_clock.cc:648] HybridClock initialized: now 1786127917642583 us; error 56 us; skew 500 ppm
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:37.646117 17293 init.cc:377] Logged in from keytab as oryx/127.6.138.125@KRBTEST.COM (short username oryx)
I20260807 18:38:37.647387 17293 webserver.cc:460] Webserver started at http://127.6.138.125:33821/ using document root <none> and password file <none>
I20260807 18:38:37.648173 17293 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:37.648313 17293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:37.648593 17293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:37.650946 17293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data/instance:
uuid: "ab5e1635aad8494e88708702bf15dab4"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "97a9a11ec04bd48c45ba8312ce50bae1"
server_key_iv: "76b16efd62986fab8dd03d2dc7ee5fe0"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.651616 17293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal/instance:
uuid: "ab5e1635aad8494e88708702bf15dab4"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "97a9a11ec04bd48c45ba8312ce50bae1"
server_key_iv: "76b16efd62986fab8dd03d2dc7ee5fe0"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.656106 17293 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260807 18:38:37.659044 17316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.660687 17293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:37.660851 17293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal
uuid: "ab5e1635aad8494e88708702bf15dab4"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "97a9a11ec04bd48c45ba8312ce50bae1"
server_key_iv: "76b16efd62986fab8dd03d2dc7ee5fe0"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.661075 17293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:37.674212 17293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:37.677888 17293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:37.678130 17293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:37.686203 17293 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.125:35473
I20260807 18:38:37.686578 17403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.125:35473 every 8 connection(s)
I20260807 18:38:37.687311 17293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
I20260807 18:38:37.688934 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 17293
I20260807 18:38:37.690991 17405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:37.695542 17405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.138.125@KRBTEST.COM
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.125@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:37.705780 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.690849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44262 (local address 127.6.138.125:35473)
0807 18:38:37.691395 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:37.691410 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:37.691448 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:38:37.692431 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:37.692450 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:37.692809 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:37.693166 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:37.694102 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.695263 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.696458 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.696942 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.697204 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.700212 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:37.700219 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:37.700231 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:37.700299 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:37.702294 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.702876 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.702881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.702886 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.703043 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.703437 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.703443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.703447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.704036 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:37.704342 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:37.704625 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 18:38:37.704939 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":168,"threads_started":1}
I20260807 18:38:37.708077 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.697703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32872 (local address 127.6.138.126:38925)
0807 18:38:37.697862 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:37.697866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:37.698349 (+ 483us) server_negotiation.cc:407] Connection header received
0807 18:38:37.698552 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:37.698555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:37.698612 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:37.698703 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:37.699707 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.700395 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.701366 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.701636 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.701758 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.704664 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:37.704667 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:37.704670 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:37.704696 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:37.706247 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.706731 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.706735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.706737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.706817 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.707111 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.707116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.707118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.707341 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:37.707462 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:37.707787 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 18:38:37.707912 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:38:37.708127 17411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.697640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:32872)
0807 18:38:37.698194 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:37.698237 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:37.698413 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:37.698760 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:37.698765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:37.699340 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:37.699528 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.699536 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.700534 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.700537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.701225 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.701234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.701763 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.701767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.701942 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.702514 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:37.702530 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:37.704503 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:37.706367 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.706376 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.706382 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.706622 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.706915 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.706918 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.706920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.707016 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.707453 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:37.707460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:37.707581 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 18:38:37.707727 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:38:37.707913 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":184,"threads_started":1}
I20260807 18:38:37.709445 17405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.125@KRBTEST.COM for oryx/127.6.138.125@KRBTEST.COM
I20260807 18:38:37.718500 17411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.709982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.0.0.1:44274)
0807 18:38:37.710099 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:37.710115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:37.710218 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:37.710444 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:37.710446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:37.710612 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:37.710691 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.710696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.711891 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.711894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.712562 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.712571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.713014 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.713017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.713103 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.713624 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:37.713636 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:37.714967 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:37.717035 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.717038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.717039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.717247 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.717510 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.717512 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.717513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.717592 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.718090 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:37.718092 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:37.718172 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 18:38:37.718261 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:38:37.718366 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 18:38:37.718717 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.710135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44274 (local address 127.6.138.125:35473)
0807 18:38:37.710256 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:37.710262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:37.710275 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:38:37.710305 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:37.710307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:37.710345 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:37.710433 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:37.710805 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.711746 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.712676 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.712896 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.713015 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.715193 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:37.715196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:37.715198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:37.715221 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:37.716865 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.717343 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.717345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.717346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.717422 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.717718 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.717721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.717723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.717985 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:37.718086 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:37.718348 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 18:38:37.718469 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:38:37.722007 17405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: Bootstrap starting.
I20260807 18:38:37.724466 17405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:37.725203 17405 log.cc:824] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:37.727181 17405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: No bootstrap required, opened a new log
I20260807 18:38:37.730266 17405 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } }
I20260807 18:38:37.730537 17405 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:37.730583 17405 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab5e1635aad8494e88708702bf15dab4, State: Initialized, Role: FOLLOWER
I20260807 18:38:37.730969 17405 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } }
I20260807 18:38:37.731544 17416 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } } }
I20260807 18:38:37.731663 17416 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:38:37.732360 17405 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:38:37.737572 17437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:38:37.737676 17437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:38:37.742956 17293 master_runner.cc:428] Detected that this master ab5e1635aad8494e88708702bf15dab4 is joining an existing cluster
I20260807 18:38:37.743036 17293 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.138.125:35473
I20260807 18:38:37.754516 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.745998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.0.0.1:44284)
0807 18:38:37.746334 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:37.746348 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:37.746473 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:37.746801 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:37.746804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:37.747098 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:37.747247 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.747257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.748339 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.748343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.749363 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.749375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.749917 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.749920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.750023 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.750453 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:37.750476 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:37.750913 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:37.752896 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.752899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.752901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.753102 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.753386 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.753388 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.753391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.753525 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.753907 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:37.753910 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:37.753995 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 18:38:37.754112 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:38:37.754275 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":2,"spinlock_wait_cycles":3328,"thread_start_us":76,"threads_started":1}
I20260807 18:38:37.754917 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.746018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44284 (local address 127.6.138.125:35473)
0807 18:38:37.746150 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:37.746155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:37.746450 (+ 295us) server_negotiation.cc:407] Connection header received
0807 18:38:37.746575 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:37.746578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:37.746621 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:37.746722 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:37.747397 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.748182 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.749517 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.749794 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.749906 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.751035 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:37.751039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:37.751041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:37.751068 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:37.752763 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.753203 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.753208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.753210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.753286 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.753623 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.753626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.753628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.753807 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:37.753918 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:37.754158 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 18:38:37.754806 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":526,"server-negotiator.queue_time_us":64}
I20260807 18:38:37.755650 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.745553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.6.138.126:38925)
0807 18:38:37.745714 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:37.745720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:37.745847 (+ 127us) server_negotiation.cc:407] Connection header received
0807 18:38:37.746014 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:37.746018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:37.746083 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:37.746149 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:37.746891 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.747656 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.749029 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.749258 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.749374 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.750936 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:37.750941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:37.750944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:37.750980 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:37.753167 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.753806 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.753811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.753817 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.753929 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.754295 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.754298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.754300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.754596 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:37.754702 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:37.755383 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 18:38:37.755511 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:38:37.756126 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.745436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:32884)
0807 18:38:37.745731 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:37.745747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:37.745874 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:37.746310 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:37.746312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:37.746589 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:37.746758 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.746766 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.747787 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.747790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.748889 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.748899 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.749382 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.749385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.749480 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.750208 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:37.750219 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:37.750756 (+ 537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:37.753377 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.753380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.753383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.753674 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.754067 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.754070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.754072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.754166 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.755085 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:37.755088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:37.755185 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 18:38:37.755787 (+ 602us) client_negotiation.cc:240] Negotiation successful
0807 18:38:37.755950 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":111,"threads_started":1}
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127916, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.126@KRBTEST.COM for oryx/127.6.138.125@KRBTEST.COM
WARNING: no policy specified for oryx/127.6.138.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.124@KRBTEST.COM" created.
I20260807 18:38:37.774634 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.761134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.0.0.1:44292)
0807 18:38:37.762146 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:37.762207 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:37.762413 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:37.762833 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:37.762841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:37.763475 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:37.763678 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.763688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.764931 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.764936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.766206 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.766220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.766923 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.766927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.767071 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.767795 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:37.767812 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:37.770028 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:37.772157 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.772171 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.772179 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.772642 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.772984 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.772987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.772989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.773107 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.773743 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:37.773753 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:37.773947 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 18:38:37.774149 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 18:38:37.774416 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":118,"threads_started":1}
I20260807 18:38:37.774672 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.761593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44292 (local address 127.6.138.125:35473)
0807 18:38:37.761731 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:37.761736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:37.762290 (+ 554us) server_negotiation.cc:407] Connection header received
0807 18:38:37.762589 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:37.762595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:37.762669 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:37.762768 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:37.763833 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.764795 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.766369 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.766772 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.766891 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.770224 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:37.770227 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:37.770230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:37.770257 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:37.771947 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.772764 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.772768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.772770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.772856 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.773222 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.773230 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.773231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.773594 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:37.773712 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:37.774247 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 18:38:37.774404 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:38:37.776957 16554 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.138.125:35473: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ab5e1635aad8494e88708702bf15dab4"
member_type: NON_VOTER
last_known_addr {
host: "127.6.138.125"
port: 35473
}
attrs {
promote: true
}
}
dest_uuid: "aecb998e302b4c218cfdbe152b4da9b1"
cas_config_opid_index: -1
I20260807 18:38:37.777751 16554 consensus_queue.cc:237] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } }
I20260807 18:38:37.779933 17452 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ab5e1635aad8494e88708702bf15dab4 (127.6.138.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } } }
I20260807 18:38:37.780820 17358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:37.781911 17452 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.138.125:35473
I20260807 18:38:37.782295 17452 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } } }
I20260807 18:38:37.782408 17452 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:37.782481 17293 master.cc:537] Master@127.6.138.125:35473 shutting down...
I20260807 18:38:37.782653 17358 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Refusing update from remote peer aecb998e302b4c218cfdbe152b4da9b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 18:38:37.783638 17452 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:38:37.785917 16530 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 18:38:37.787338 16530 consensus_peers.cc:487] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 -> Peer ab5e1635aad8494e88708702bf15dab4 (127.6.138.125:35473): Couldn't send request to peer ab5e1635aad8494e88708702bf15dab4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:38:37.788295 17293 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:37.788647 17293 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:37.788781 17293 tablet_replica.cc:330] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.124.keytab.
Entry for principal oryx/127.6.138.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.124.keytab.
I20260807 18:38:37.801033 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.124:38831
--webserver_interface=127.6.138.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.124.keytab
--principal=oryx/127.6.138.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:38925,127.6.138.125:35473,127.6.138.124:38831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:38:37.803870 17293 master.cc:552] Master@127.6.138.125:35473 shutdown complete.
I20260807 18:38:37.803941 17293 master_runner.cc:305] Clearing existing system tablet
I20260807 18:38:37.804224 17293 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:37.804320 17293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:37.805779 17293 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:37.806808 17462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.807075 17293 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:37.807242 17293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:37.807292 17293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal
uuid: "ab5e1635aad8494e88708702bf15dab4"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "97a9a11ec04bd48c45ba8312ce50bae1"
server_key_iv: "76b16efd62986fab8dd03d2dc7ee5fe0"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.810276 17293 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:37.813669 17293 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:38:37.813799 17293 log.cc:1196] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 18:38:37.814121 17293 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: Deleting consensus metadata
I20260807 18:38:37.814327 17293 master_runner.cc:315] Copying system tablet from 127.6.138.126:38925
I20260807 18:38:37.815939 17293 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: tablet copy: Beginning tablet copy session from remote peer at address 127.6.138.126:38925
I20260807 18:38:37.825853 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.816780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32898 (local address 127.6.138.126:38925)
0807 18:38:37.816914 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:37.816920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:37.817132 (+ 212us) server_negotiation.cc:407] Connection header received
0807 18:38:37.817251 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:37.817254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:37.817308 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:37.817420 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:37.818003 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.818815 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.820095 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.820469 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.820587 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.822031 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:37.822036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:37.822038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:37.822068 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:37.823914 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.824469 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.824473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.824475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.824569 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:37.824877 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:37.824880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:37.824882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:37.825123 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:37.825226 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:37.825532 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 18:38:37.825663 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:38:37.825876 17472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:37.816662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:32898)
0807 18:38:37.817021 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:37.817038 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:37.817163 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:37.817459 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:37.817465 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:37.817721 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:37.817862 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.817869 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.818944 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.818948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.819952 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:37.819963 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:37.820600 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:37.820604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:37.820706 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:37.821325 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:37.821341 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:37.821853 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:37.824051 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.824054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.824057 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.824358 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.824673 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:37.824676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:37.824678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:37.824779 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:37.825227 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:37.825230 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:37.825343 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 18:38:37.825460 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:38:37.825626 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":99,"threads_started":1}
I20260807 18:38:37.827055 16588 tablet_copy_service.cc:140] P aecb998e302b4c218cfdbe152b4da9b1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ab5e1635aad8494e88708702bf15dab4 ({username='oryx', principal='oryx/127.6.138.125@KRBTEST.COM'} at 127.0.0.1:32898)
I20260807 18:38:37.827268 16588 tablet_copy_service.cc:161] P aecb998e302b4c218cfdbe152b4da9b1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ab5e1635aad8494e88708702bf15dab4 at {username='oryx', principal='oryx/127.6.138.125@KRBTEST.COM'} at 127.0.0.1:32898: session id = ab5e1635aad8494e88708702bf15dab4-00000000000000000000000000000000
I20260807 18:38:37.829408 16588 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:38:37.831651 17293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:38:37.834398 17293 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: tablet copy: Starting download of 0 data blocks...
I20260807 18:38:37.834626 17293 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: tablet copy: Starting download of 1 WAL segments...
I20260807 18:38:37.836716 17293 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:38:37.838742 16588 tablet_copy_service.cc:342] P aecb998e302b4c218cfdbe152b4da9b1: Request end of tablet copy session ab5e1635aad8494e88708702bf15dab4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.138.125@KRBTEST.COM'} at 127.0.0.1:32898
I20260807 18:38:37.838891 16588 tablet_copy_service.cc:433] P aecb998e302b4c218cfdbe152b4da9b1: ending tablet copy session ab5e1635aad8494e88708702bf15dab4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ab5e1635aad8494e88708702bf15dab4
I20260807 18:38:37.846863 17293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:37.849010 17473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.849347 17476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.849409 17474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:37.849579 17293 server_base.cc:1034] running on GCE node
I20260807 18:38:37.849892 17293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:37.850241 17293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:37.851399 17293 hybrid_clock.cc:648] HybridClock initialized: now 1786127917851381 us; error 37 us; skew 500 ppm
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:37.853046 17293 init.cc:377] Logged in from keytab as oryx/127.6.138.125@KRBTEST.COM (short username oryx)
I20260807 18:38:37.853528 17293 webserver.cc:460] Webserver started at http://127.6.138.125:33505/ using document root <none> and password file <none>
I20260807 18:38:37.853722 17293 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:37.853765 17293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:37.855398 17293 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 18:38:37.856506 17485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.856810 17293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:37.856892 17293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal
uuid: "ab5e1635aad8494e88708702bf15dab4"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "97a9a11ec04bd48c45ba8312ce50bae1"
server_key_iv: "76b16efd62986fab8dd03d2dc7ee5fe0"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.856987 17293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:37.874148 17293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:37.876598 17293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:37.882196 17293 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.125:35473
I20260807 18:38:37.882344 17571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.125:35473 every 8 connection(s)
I20260807 18:38:37.882936 17293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
I20260807 18:38:37.885133 17572 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:38:37.886250 17572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: Bootstrap starting.
I20260807 18:38:37.901216 17572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:38:37.901611 17572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: Bootstrap complete.
I20260807 18:38:37.902302 17572 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } }
I20260807 18:38:37.902482 17572 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ab5e1635aad8494e88708702bf15dab4, State: Initialized, Role: LEARNER
I20260807 18:38:37.902607 17572 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } }
I20260807 18:38:37.903026 17578 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } } }
I20260807 18:38:37.903188 17572 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:37.903278 17578 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:38:37.906747 17591 catalog_manager.cc:1259] Loaded cluster ID: 589dd4e6265641c3bb2a26ad1e6a6187
I20260807 18:38:37.906922 17591 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: loading cluster ID for follower catalog manager: success
I20260807 18:38:37.909569 17591 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: acquiring CA information for follower catalog manager: success
I20260807 18:38:37.911422 17591 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:38:37.947515 17457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:37.947891 17457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:37.947978 17457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:37.952982 17457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:37.953074 17457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:37.953097 17457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:37.953116 17457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:37.953171 17457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:37.959867 17457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:38925,127.6.138.125:35473,127.6.138.124:38831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.124.keytab
--principal=oryx/127.6.138.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.124:38831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.138.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:37.961390 17457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:37.962749 17457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:38:37.967643 16857 heartbeater.cc:503] Master 127.6.138.126:38925 was elected leader, sending a full tablet report...
W20260807 18:38:37.973147 17601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.973147 17599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:37.973260 17598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:37.973423 17457 server_base.cc:1034] running on GCE node
I20260807 18:38:37.974124 17457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:37.975425 17457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:37.976617 17457 hybrid_clock.cc:648] HybridClock initialized: now 1786127917976600 us; error 32 us; skew 500 ppm
Aug 07 18:38:37 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:37.979871 17457 init.cc:377] Logged in from keytab as oryx/127.6.138.124@KRBTEST.COM (short username oryx)
I20260807 18:38:37.981012 17457 webserver.cc:460] Webserver started at http://127.6.138.124:32861/ using document root <none> and password file <none>
I20260807 18:38:37.981628 17457 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:37.981726 17457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:37.981986 17457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:37.984364 17457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data/instance:
uuid: "a9b8b0ac167c4d338f13c995340ab883"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "37bdae1630a0a286a9fe3b8eebf27bf9"
server_key_iv: "672eb9737cdb4bf3692081fd986a175b"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.985013 17457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal/instance:
uuid: "a9b8b0ac167c4d338f13c995340ab883"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "37bdae1630a0a286a9fe3b8eebf27bf9"
server_key_iv: "672eb9737cdb4bf3692081fd986a175b"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.989635 17457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:38:37.993026 17615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:37.994807 17457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:38:37.995009 17457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal
uuid: "a9b8b0ac167c4d338f13c995340ab883"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "37bdae1630a0a286a9fe3b8eebf27bf9"
server_key_iv: "672eb9737cdb4bf3692081fd986a175b"
server_key_version: "encryptionkey@0"
I20260807 18:38:37.995162 17457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:38.030550 17457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:38.034543 17457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:38.034794 17457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:38.043126 17457 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.124:38831
I20260807 18:38:38.043427 17710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.124:38831 every 8 connection(s)
I20260807 18:38:38.044317 17457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
I20260807 18:38:38.047384 17712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:38.050631 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 17457
I20260807 18:38:38.051873 17712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.124@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.138.124@KRBTEST.COM
I20260807 18:38:38.066677 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.054334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.6.138.126:38925)
0807 18:38:38.054487 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.054492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.054755 (+ 263us) server_negotiation.cc:407] Connection header received
0807 18:38:38.054891 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.054894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.054956 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.055062 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.056563 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.057121 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.058532 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.058795 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.058903 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.061775 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.061778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.061780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.061807 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.063823 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.064580 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.064583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.064585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.064719 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.065113 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.065120 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.065126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.065429 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.065552 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.066268 (+ 716us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.066471 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:38:38.067621 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.054132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:32914)
0807 18:38:38.054587 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.054636 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.054767 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.055146 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.055158 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.055661 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.056391 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.056404 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.057328 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.057332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.058336 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.058345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.058972 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.058976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.059099 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.059796 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.059813 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.061575 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.063968 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.063978 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.063989 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.064465 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.064835 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.064838 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.064839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.064982 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.065574 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.065581 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.065912 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.066177 (+ 265us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.066584 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":63,"thread_start_us":156,"threads_started":1}
I20260807 18:38:38.067634 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.052619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59190 (local address 127.6.138.124:38831)
0807 18:38:38.053185 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.053205 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.053248 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:38:38.054164 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.054194 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.054668 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.055984 (+ 1316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.056155 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.057342 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.058330 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.058823 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.059013 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.062393 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.062402 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.062419 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.062452 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.064550 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.065164 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.065169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.065170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.065241 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.065573 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.065576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.065578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.065914 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.066105 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.066387 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.066742 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":197,"server-negotiator.queue_time_us":414,"thread_start_us":176,"threads_started":1}
I20260807 18:38:38.069772 17712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.124@KRBTEST.COM for oryx/127.6.138.125@KRBTEST.COM
I20260807 18:38:38.078642 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.070389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.0.0.1:44308)
0807 18:38:38.070531 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.070545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.070632 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.071115 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.071118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.071267 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.071380 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.071386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.072410 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.072413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.072957 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.072965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.073430 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.073434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.073528 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.074052 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.074069 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.075344 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.077261 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.077264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.077266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.077471 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.077735 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.077738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.077739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.077816 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.078218 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.078220 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.078306 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.078391 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.078505 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 18:38:38.078708 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.070522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44308 (local address 127.6.138.125:35473)
0807 18:38:38.070846 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.070850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.070864 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:38:38.070903 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.070905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.070982 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.071071 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.071503 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.072239 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.073063 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.073290 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.073382 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.075490 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.075494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.075497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.075525 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.077154 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.077564 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.077567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.077568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.077645 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.077922 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.077925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.077926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.078128 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.078208 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.078476 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.078582 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":115,"threads_started":1}
I20260807 18:38:38.079619 17712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127917, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.124@KRBTEST.COM for oryx/127.6.138.124@KRBTEST.COM
I20260807 18:38:38.088544 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.080168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.0.0.1:59192)
0807 18:38:38.080341 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.080358 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.080444 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.080684 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.080687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.080854 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.080916 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.080921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.081736 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.081743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.082242 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.082249 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.083267 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.083270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.083356 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.083862 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.083876 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.085263 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.087124 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.087127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.087129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.087364 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.087641 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.087643 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.087645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.087720 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.088050 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.088052 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.088130 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.088208 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.088397 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":3}
I20260807 18:38:38.088665 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.080218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.6.138.124:38831)
0807 18:38:38.080341 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.080347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.080402 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:38:38.080546 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.080549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.080592 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.080672 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.081071 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.081618 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.082357 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.083142 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.083269 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.085426 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.085430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.085432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.085490 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.087013 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.087481 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.087483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.087485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.087559 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.087797 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.087799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.087800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.087972 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.088045 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.088323 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.088485 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":67}
I20260807 18:38:38.091805 17712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Bootstrap starting.
I20260807 18:38:38.094481 17712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:38.095279 17712 log.cc:824] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:38.097586 17712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: No bootstrap required, opened a new log
I20260807 18:38:38.100983 17712 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } }
I20260807 18:38:38.101193 17712 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:38.101265 17712 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b8b0ac167c4d338f13c995340ab883, State: Initialized, Role: FOLLOWER
I20260807 18:38:38.101742 17712 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } }
I20260807 18:38:38.102277 17728 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } } }
I20260807 18:38:38.102487 17728 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:38:38.103076 17712 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:38:38.108639 17742 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:38:38.108752 17742 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:38:38.112547 17457 master_runner.cc:428] Detected that this master a9b8b0ac167c4d338f13c995340ab883 is joining an existing cluster
I20260807 18:38:38.112640 17457 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.138.124:38831
I20260807 18:38:38.121523 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 16656
I20260807 18:38:38.132584 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.115932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.6.138.125:35473)
0807 18:38:38.116070 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.116075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.120585 (+ 4510us) server_negotiation.cc:407] Connection header received
0807 18:38:38.120813 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.120828 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.120893 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.120954 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.122400 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.123170 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.124429 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.125009 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.125129 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.127078 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.127085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.127092 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.127131 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.129588 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.130352 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.130360 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.130365 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.130487 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.130961 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.130968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.130973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.131334 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.131454 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.132033 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.132245 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:38:38.133952 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.116078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.0.0.1:44322)
0807 18:38:38.120483 (+ 4405us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.120499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.120682 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.121498 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.121501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.121774 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.122252 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.122264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.123315 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.123321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.124241 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.124273 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.125370 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.125377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.125529 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.126206 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.126230 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.126852 (+ 622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.129772 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.129779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.129784 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.130178 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.130654 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.130661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.130666 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.130807 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.131538 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.131545 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.131704 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.133334 (+ 1630us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.133595 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4319,"thread_start_us":121,"threads_started":1}
I20260807 18:38:38.134657 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.115264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.6.138.126:38925)
0807 18:38:38.115415 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.115420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.115730 (+ 310us) server_negotiation.cc:407] Connection header received
0807 18:38:38.115976 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.115980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.116039 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.116129 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.116870 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.117483 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.118840 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.119193 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.119272 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.120898 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.120902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.120905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.120932 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.130360 (+ 9428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.132703 (+ 2343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.132706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.132708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.132794 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.133072 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.133075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.133076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.133259 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.133325 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.134401 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.134528 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:38:38.136693 17750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.115278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:32916)
0807 18:38:38.115624 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.115644 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.115824 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.116303 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.116306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.116570 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.116721 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.116729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.117609 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.117613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.118640 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.118652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.119437 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.119440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.119567 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.120228 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.120241 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.120739 (+ 498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.132223 (+ 11484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.132226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.132228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.132482 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.132891 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.132894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.132895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.132964 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.133569 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.133571 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.133646 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.135117 (+ 1471us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.135253 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":27,"thread_start_us":156,"threads_started":1}
I20260807 18:38:38.138163 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.115524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59208 (local address 127.6.138.124:38831)
0807 18:38:38.115632 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.115636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.116536 (+ 900us) server_negotiation.cc:407] Connection header received
0807 18:38:38.116697 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.116700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.116740 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.116850 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.118093 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.118648 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.119544 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.119851 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.119973 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.134089 (+ 14116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.134092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.134094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.134110 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.135978 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.136360 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.136363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.136364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.136417 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.136650 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.136658 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.136663 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.137066 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.137209 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.137679 (+ 470us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.137864 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":45}
I20260807 18:38:38.139508 17757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.115698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.0.0.1:59208)
0807 18:38:38.116432 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.116445 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.116577 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.117626 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.117629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.117850 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.117949 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.117954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.118769 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.118775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.119399 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.119408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.119981 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.119985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.120089 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.133383 (+ 13294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.133395 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.133975 (+ 580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.136083 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.136085 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.136087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.136286 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.136498 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.136500 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.136502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.136565 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.137269 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.137276 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.137426 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.138348 (+ 922us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.138561 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":101,"threads_started":1}
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127916, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.126@KRBTEST.COM for oryx/127.6.138.124@KRBTEST.COM
I20260807 18:38:38.157760 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:43745
--local_ip_for_outbound_sockets=127.6.138.65
--tserver_master_addrs=127.6.138.126:38925
--webserver_port=37763
--webserver_interface=127.6.138.65
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab
--principal=oryx/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.138.126:38925,127.6.138.125:35473
--tserver_master_addrs=127.6.138.126:38925,127.6.138.125:35473,127.6.138.124:38831 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:38:38.158514 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.146331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.0.0.1:59222)
0807 18:38:38.146529 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.146550 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.146663 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.147309 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.147315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.147589 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.147762 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.147774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.149166 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.149174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.150317 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.150337 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.151052 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.151059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.151214 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.151928 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.151952 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.153934 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.156552 (+ 2618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.156556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.156559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.156855 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.157229 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.157233 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.157236 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.157329 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.157909 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.157913 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.158032 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.158151 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.158318 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:38:38.159212 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.146839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.6.138.124:38831)
0807 18:38:38.146995 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.147001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.147017 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:38.147106 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.147112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.147177 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.147910 (+ 733us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.148033 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.149003 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.150490 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.150882 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.151010 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.154129 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.154137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.154144 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.154185 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.156380 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.157006 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.157010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.157012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.157114 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.157464 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.157472 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.157476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.157777 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.157906 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.158523 (+ 617us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.158948 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:38:38.160733 16554 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.138.124:38831: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883"
member_type: NON_VOTER
last_known_addr {
host: "127.6.138.124"
port: 38831
}
attrs {
promote: true
}
}
dest_uuid: "aecb998e302b4c218cfdbe152b4da9b1"
cas_config_opid_index: 5
I20260807 18:38:38.161576 16554 consensus_queue.cc:237] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } }
I20260807 18:38:38.162923 17452 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a9b8b0ac167c4d338f13c995340ab883 (127.6.138.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.163537 17668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:38.164644 17452 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.138.124:38831
I20260807 18:38:38.165004 17668 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Refusing update from remote peer aecb998e302b4c218cfdbe152b4da9b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 18:38:38.165126 17457 master.cc:537] Master@127.6.138.124:38831 shutting down...
I20260807 18:38:38.165148 17452 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.165259 17452 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:38.165905 17453 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:38:38.168403 16531 consensus_peers.cc:487] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 -> Peer a9b8b0ac167c4d338f13c995340ab883 (127.6.138.124:38831): Couldn't send request to peer a9b8b0ac167c4d338f13c995340ab883. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:38:38.178277 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.166921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.0.0.1:44328)
0807 18:38:38.169718 (+ 2797us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.169736 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.170178 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.170640 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.170643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.170665 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:38.170781 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.170788 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.172058 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.172062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.173597 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.173610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.177697 (+ 4087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.177702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.177825 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.177910 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.178013 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.178109 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2702}
I20260807 18:38:38.178851 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.167174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44328 (local address 127.6.138.125:35473)
0807 18:38:38.167306 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.167311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.169811 (+ 2500us) server_negotiation.cc:407] Connection header received
0807 18:38:38.170341 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.170348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.170455 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.170593 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:38.170926 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.171909 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.176526 (+ 4617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.177513 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.177682 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.177889 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.178627 (+ 738us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.178724 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:38:38.179906 17534 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 LEARNER]: Refusing update from remote peer aecb998e302b4c218cfdbe152b4da9b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 18:38:38.180567 17760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 18:38:38.180847 17457 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:38.181224 17457 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:38.181358 17457 tablet_replica.cc:330] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: stopping tablet replica
I20260807 18:38:38.182936 17534 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a9b8b0ac167c4d338f13c995340ab883 (127.6.138.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.185750 17578 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aecb998e302b4c218cfdbe152b4da9b1. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.185913 17578 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:38:38.186019 17578 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: NON_VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: true } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.186091 17578 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:38:38.187125 17453 raft_consensus.cc:1050] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: attempting to promote NON_VOTER ab5e1635aad8494e88708702bf15dab4 to VOTER
I20260807 18:38:38.187732 17457 master.cc:552] Master@127.6.138.124:38831 shutdown complete.
I20260807 18:38:38.187649 17453 consensus_queue.cc:237] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } }
I20260807 18:38:38.187785 17457 master_runner.cc:305] Clearing existing system tablet
I20260807 18:38:38.188068 17457 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:38.188216 17457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:38.189198 17532 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 LEARNER]: Refusing update from remote peer aecb998e302b4c218cfdbe152b4da9b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 18:38:38.189736 17762 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 18:38:38.190577 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.188811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.0.0.1:59238)
0807 18:38:38.190369 (+ 1558us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.190486 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1463}
I20260807 18:38:38.191500 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.191086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.0.0.1:59252)
0807 18:38:38.191354 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.191404 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":93,"threads_started":1}
W20260807 18:38:38.191654 16531 consensus_peers.cc:487] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 -> Peer a9b8b0ac167c4d338f13c995340ab883 (127.6.138.124:38831): Couldn't send request to peer a9b8b0ac167c4d338f13c995340ab883. Status: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 18:38:38.192037 17762 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ab5e1635aad8494e88708702bf15dab4 (127.6.138.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.192587 17532 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ab5e1635aad8494e88708702bf15dab4 (127.6.138.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.193804 17762 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.193915 17762 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:38.194039 17762 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.194109 17762 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:38.194635 17764 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.194767 17764 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:38:38.195583 17457 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260807 18:38:38.196866 17769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.197140 17457 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:38.197294 17457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 18:38:38.197348 17457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal
uuid: "a9b8b0ac167c4d338f13c995340ab883"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "37bdae1630a0a286a9fe3b8eebf27bf9"
server_key_iv: "672eb9737cdb4bf3692081fd986a175b"
server_key_version: "encryptionkey@0"
I20260807 18:38:38.199759 17457 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:38.201138 17074 heartbeater.cc:503] Master 127.6.138.126:38925 was elected leader, sending a full tablet report...
I20260807 18:38:38.202467 17457 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:38:38.202570 17457 log.cc:1196] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 18:38:38.202898 17457 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Deleting consensus metadata
I20260807 18:38:38.203121 17457 master_runner.cc:315] Copying system tablet from 127.6.138.126:38925
I20260807 18:38:38.204583 17457 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: tablet copy: Beginning tablet copy session from remote peer at address 127.6.138.126:38925
I20260807 18:38:38.220064 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.205428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32928 (local address 127.6.138.126:38925)
0807 18:38:38.205574 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.205578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.210855 (+ 5277us) server_negotiation.cc:407] Connection header received
0807 18:38:38.211043 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.211046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.211105 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.211199 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.211801 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.212749 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.214043 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.214443 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.214572 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.216014 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.216019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.216022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.216054 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.218054 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.218652 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.218656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.218658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.218755 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.219110 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.219113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.219115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.219335 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.219435 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.219771 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.219895 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:38:38.220117 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.205213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:32928)
0807 18:38:38.210728 (+ 5515us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.210744 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.210913 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.211259 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.211263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.211521 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.211666 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.211673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.212892 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.212896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.213926 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.213939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.214587 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.214591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.214694 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.215360 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.215376 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.215839 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.218204 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.218208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.218210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.218531 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.218871 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.218874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.218879 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.219003 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.219449 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.219452 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.219572 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.219697 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.219861 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5428,"thread_start_us":91,"threads_started":1}
I20260807 18:38:38.220898 16588 tablet_copy_service.cc:140] P aecb998e302b4c218cfdbe152b4da9b1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a9b8b0ac167c4d338f13c995340ab883 ({username='oryx', principal='oryx/127.6.138.124@KRBTEST.COM'} at 127.0.0.1:32928)
I20260807 18:38:38.221031 16588 tablet_copy_service.cc:161] P aecb998e302b4c218cfdbe152b4da9b1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a9b8b0ac167c4d338f13c995340ab883 at {username='oryx', principal='oryx/127.6.138.124@KRBTEST.COM'} at 127.0.0.1:32928: session id = a9b8b0ac167c4d338f13c995340ab883-00000000000000000000000000000000
I20260807 18:38:38.222323 16588 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:38:38.223752 17457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:38:38.226327 17457 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: tablet copy: Starting download of 0 data blocks...
I20260807 18:38:38.226544 17457 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: tablet copy: Starting download of 1 WAL segments...
I20260807 18:38:38.228359 17457 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:38:38.230468 16588 tablet_copy_service.cc:342] P aecb998e302b4c218cfdbe152b4da9b1: Request end of tablet copy session a9b8b0ac167c4d338f13c995340ab883-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.138.124@KRBTEST.COM'} at 127.0.0.1:32928
I20260807 18:38:38.230564 16588 tablet_copy_service.cc:433] P aecb998e302b4c218cfdbe152b4da9b1: ending tablet copy session a9b8b0ac167c4d338f13c995340ab883-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a9b8b0ac167c4d338f13c995340ab883
I20260807 18:38:38.239269 17457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:38.241091 17781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:38.241447 17779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:38.244472 17778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:38.245245 17457 server_base.cc:1034] running on GCE node
I20260807 18:38:38.245426 17457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:38.245748 17457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:38.246884 17457 hybrid_clock.cc:648] HybridClock initialized: now 1786127918246865 us; error 47 us; skew 500 ppm
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:38.248646 17457 init.cc:377] Logged in from keytab as oryx/127.6.138.124@KRBTEST.COM (short username oryx)
I20260807 18:38:38.249315 17457 webserver.cc:460] Webserver started at http://127.6.138.124:38453/ using document root <none> and password file <none>
I20260807 18:38:38.249522 17457 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:38.249564 17457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:38.250952 17457 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:38.252107 17792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.252414 17457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:38.252534 17457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal
uuid: "a9b8b0ac167c4d338f13c995340ab883"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "37bdae1630a0a286a9fe3b8eebf27bf9"
server_key_iv: "672eb9737cdb4bf3692081fd986a175b"
server_key_version: "encryptionkey@0"
I20260807 18:38:38.252661 17457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:38.265733 17457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:38.268174 17457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:38.273772 17457 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.124:38831
I20260807 18:38:38.273810 17868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.124:38831 every 8 connection(s)
I20260807 18:38:38.274641 17457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
I20260807 18:38:38.276798 17869 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:38:38.277944 17869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Bootstrap starting.
I20260807 18:38:38.292037 17869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:38:38.292495 17869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Bootstrap complete.
I20260807 18:38:38.293184 17869 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } }
I20260807 18:38:38.293404 17869 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a9b8b0ac167c4d338f13c995340ab883, State: Initialized, Role: LEARNER
I20260807 18:38:38.293541 17869 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } }
I20260807 18:38:38.293938 17872 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.294092 17869 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:38.294178 17872 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:38:38.298213 17885 catalog_manager.cc:1259] Loaded cluster ID: 589dd4e6265641c3bb2a26ad1e6a6187
I20260807 18:38:38.298282 17885 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: loading cluster ID for follower catalog manager: success
I20260807 18:38:38.300379 17885 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: acquiring CA information for follower catalog manager: success
I20260807 18:38:38.301692 17885 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:38:38.329698 17759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:38.330054 17759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:38.330164 17759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:38.334657 17759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:38.334767 17759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:38.334874 17759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:38.340821 17759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab
--principal=oryx/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:43745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=37763
--tserver_master_addrs=127.6.138.126:38925,127.6.138.125:35473,127.6.138.124:38831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:38.341984 17759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:38.343075 17759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:38.353654 17892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:38.353658 17893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:38.353654 17895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:38.354049 17759 server_base.cc:1034] running on GCE node
I20260807 18:38:38.354524 17759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:38.355158 17759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:38.356346 17759 hybrid_clock.cc:648] HybridClock initialized: now 1786127918356307 us; error 49 us; skew 500 ppm
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:38.359652 17759 init.cc:377] Logged in from keytab as oryx/127.6.138.65@KRBTEST.COM (short username oryx)
I20260807 18:38:38.360781 17759 webserver.cc:460] Webserver started at http://127.6.138.65:37763/ using document root <none> and password file <none>
I20260807 18:38:38.361353 17759 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:38.361431 17759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:38.365494 17759 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:38:38.367861 17907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.370221 17759 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 18:38:38.370381 17759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "52badaa1b1204ab1b901c2d11e7e1a79"
format_stamp: "Formatted at 2026-08-07 18:38:36 on dist-test-slave-2d19"
server_key: "5b4fa31a80d29560681574e7aee21e3c"
server_key_iv: "39d3903ea9f2353ee3babf8621dcaf03"
server_key_version: "encryptionkey@0"
I20260807 18:38:38.371178 17759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:38.386554 17759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:38.390451 17759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:38.390650 17759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:38.391376 17759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:38.392578 17759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:38.392644 17759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.392716 17759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:38.392751 17759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.403162 17759 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:43745
I20260807 18:38:38.403200 18074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:43745 every 8 connection(s)
I20260807 18:38:38.404572 17759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:38.411926 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 17759
I20260807 18:38:38.412201 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 16866
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.124@KRBTEST.COM
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.125@KRBTEST.COM
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:38.424139 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.408347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:56221 (local address 127.6.138.125:35473)
0807 18:38:38.408499 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.408503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.408613 (+ 110us) server_negotiation.cc:407] Connection header received
0807 18:38:38.409698 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.409701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.409761 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.409858 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.410893 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.411735 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.413407 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.413771 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.413878 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.418923 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.418926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.418929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.418955 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.421205 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.421986 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.421990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.421992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.422102 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.422662 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.422665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.422667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.422899 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.422994 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.423824 (+ 830us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.423975 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:38:38.424156 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.406659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:51961 (local address 127.6.138.124:38831)
0807 18:38:38.408176 (+ 1517us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.408180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.408197 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:38:38.409365 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.409369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.409426 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.409528 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.410876 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.412000 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.414233 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.414604 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.414702 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.418916 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.418920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.418923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.418963 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.421147 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.421986 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.421990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.421992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.422230 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.422662 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.422665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.422667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.422899 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.422988 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.423824 (+ 836us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.423971 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1436,"thread_start_us":142,"threads_started":1}
I20260807 18:38:38.426028 18082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.407119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.6.138.65:51961)
0807 18:38:38.407634 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.407690 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.408800 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.409560 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.409563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.410100 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.410702 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.410719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.412139 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.412141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.414106 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.414115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.414827 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.414831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.414928 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.415474 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.415497 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.418686 (+ 3189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.421309 (+ 2623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.421315 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.421332 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.421849 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.422341 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.422344 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.422346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.422542 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.423066 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.423073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.423504 (+ 431us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.424415 (+ 911us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.424851 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":1352,"spinlock_wait_cycles":65536,"thread_start_us":153,"threads_started":1}
I20260807 18:38:38.426028 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.408200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.6.138.65:56221)
0807 18:38:38.408514 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.408528 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.408800 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.409899 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.409902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.410188 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.410702 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.410719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.411895 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.411898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.413269 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.413280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.414001 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.414011 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.414337 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.415478 (+ 1141us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.415514 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.418686 (+ 3172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.421317 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.421320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.421332 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.421849 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.422373 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.422377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.422382 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.422541 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.423066 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.423073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.423504 (+ 431us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.424683 (+ 1179us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.424906 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":366,"spinlock_wait_cycles":31872,"thread_start_us":127,"threads_started":1}
I20260807 18:38:38.427021 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:44067
--local_ip_for_outbound_sockets=127.6.138.66
--tserver_master_addrs=127.6.138.126:38925
--webserver_port=43833
--webserver_interface=127.6.138.66
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab
--principal=oryx/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.138.126:38925,127.6.138.125:35473
--tserver_master_addrs=127.6.138.126:38925,127.6.138.125:35473,127.6.138.124:38831 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:38:38.427613 18075 heartbeater.cc:347] Connected to a master server at 127.6.138.124:38831
I20260807 18:38:38.427865 18075 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:38.428481 18075 heartbeater.cc:511] Master 127.6.138.124:38831 requested a full tablet report, sending...
I20260807 18:38:38.429520 18077 heartbeater.cc:347] Connected to a master server at 127.6.138.125:35473
I20260807 18:38:38.429627 18077 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:38.429863 18077 heartbeater.cc:511] Master 127.6.138.125:35473 requested a full tablet report, sending...
I20260807 18:38:38.430580 17813 ts_manager.cc:194] Registered new tserver with Master: 52badaa1b1204ab1b901c2d11e7e1a79 (127.6.138.65:43745)
I20260807 18:38:38.431133 17519 ts_manager.cc:194] Registered new tserver with Master: 52badaa1b1204ab1b901c2d11e7e1a79 (127.6.138.65:43745)
I20260807 18:38:38.433806 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.406697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:49071 (local address 127.6.138.126:38925)
0807 18:38:38.406840 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.406845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.407801 (+ 956us) server_negotiation.cc:407] Connection header received
0807 18:38:38.409052 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.409055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.409123 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.409209 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.410866 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.411595 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.415073 (+ 3478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.415402 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.415490 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.429019 (+ 13529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.429024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.429028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.429067 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.431195 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.432150 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.432153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.432154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.432337 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.432704 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.432707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.432708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.432905 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.433002 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.433439 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.433631 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:38:38.433866 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.407119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.65:49071)
0807 18:38:38.407635 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.407692 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.408809 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.409391 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.409401 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.410093 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.410695 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.410712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.411806 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.411810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.414910 (+ 3100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.414918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.415654 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.415661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.415766 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.416320 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.416340 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.418711 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.431584 (+ 12873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.431591 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.431596 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.431959 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.432457 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.432464 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.432468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.432588 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.433028 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.433034 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.433183 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.433333 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.433548 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":1982,"spinlock_wait_cycles":29184,"thread_start_us":159,"threads_started":1}
I20260807 18:38:38.434912 18076 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38925
I20260807 18:38:38.435077 18076 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:38.435281 18076 heartbeater.cc:511] Master 127.6.138.126:38925 requested a full tablet report, sending...
I20260807 18:38:38.436373 16554 ts_manager.cc:194] Re-registered known tserver with Master: 52badaa1b1204ab1b901c2d11e7e1a79 (127.6.138.65:43745)
I20260807 18:38:38.437394 16554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:49071
I20260807 18:38:38.443956 17288 heartbeater.cc:503] Master 127.6.138.126:38925 was elected leader, sending a full tablet report...
W20260807 18:38:38.563827 18090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:38.564136 18090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:38.564226 18090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:38.568961 18090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:38.569032 18090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:38.569133 18090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:38.574795 18090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab
--principal=oryx/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:44067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=43833
--tserver_master_addrs=127.6.138.126:38925,127.6.138.125:35473,127.6.138.124:38831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:38.576098 18090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:38.577363 18090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:38.587582 18104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:38.587600 18099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:38.587582 18100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:38.588176 18090 server_base.cc:1034] running on GCE node
I20260807 18:38:38.588689 18090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:38.589522 18090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:38.590711 18090 hybrid_clock.cc:648] HybridClock initialized: now 1786127918590688 us; error 33 us; skew 500 ppm
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:38.594220 18090 init.cc:377] Logged in from keytab as oryx/127.6.138.66@KRBTEST.COM (short username oryx)
I20260807 18:38:38.595248 18090 webserver.cc:460] Webserver started at http://127.6.138.66:43833/ using document root <none> and password file <none>
I20260807 18:38:38.595942 18090 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:38.596056 18090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:38.600318 18090 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:38:38.602578 18113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.604111 18090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:38.604250 18090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "59751f6a7203483c9bdb4c47feda63ab"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "8bcc67eddb9e1ff7c9a9468c2e7fa2b9"
server_key_iv: "4edaecf47d130941de96014e48aa8ee5"
server_key_version: "encryptionkey@0"
I20260807 18:38:38.604974 18090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:38.620802 18090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:38.624114 18090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:38.624359 18090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:38.625008 18090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:38.625926 18090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:38.626005 18090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.626075 18090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:38.626125 18090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.636198 18090 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:44067
I20260807 18:38:38.636216 18265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:44067 every 8 connection(s)
I20260807 18:38:38.637383 18090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:38.637584 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 18090
I20260807 18:38:38.637766 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 17079
I20260807 18:38:38.654510 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:43671
--local_ip_for_outbound_sockets=127.6.138.67
--tserver_master_addrs=127.6.138.126:38925
--webserver_port=41225
--webserver_interface=127.6.138.67
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab
--principal=oryx/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.138.126:38925,127.6.138.125:35473
--tserver_master_addrs=127.6.138.126:38925,127.6.138.125:35473,127.6.138.124:38831 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.125@KRBTEST.COM
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.124@KRBTEST.COM
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:38.663012 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.648316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:35267 (local address 127.6.138.125:35473)
0807 18:38:38.648760 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.648765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.649032 (+ 267us) server_negotiation.cc:407] Connection header received
0807 18:38:38.649208 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.649214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.649269 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.649325 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.650657 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.651590 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.653546 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.653882 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.653972 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.658874 (+ 4902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.658879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.658884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.658921 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.660952 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.661550 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.661556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.661561 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.661685 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.661997 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.662000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.662002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.662239 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.662319 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.662689 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.662823 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361}
I20260807 18:38:38.663355 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.648607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:56823 (local address 127.6.138.126:38925)
0807 18:38:38.648723 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.648727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.648743 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:38.648790 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.648793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.648856 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.648962 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.650654 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.651504 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.653424 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.653776 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.653891 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.659323 (+ 5432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.659329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.659332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.659362 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.661512 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.662013 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.662017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.662018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.662099 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.662410 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.662413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.662414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.662641 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.662719 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.663079 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.663211 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 18:38:38.664127 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.645308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.6.138.66:54311)
0807 18:38:38.646859 (+ 1551us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.646921 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.648159 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.648896 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.648903 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.649694 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.650446 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.650464 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.651708 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.651712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.654167 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.654179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.654814 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.654818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.654925 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.655407 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.655421 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.657866 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.660136 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.660145 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.660166 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.660577 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.660949 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.660955 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.660960 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.661203 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.661785 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.661795 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.662183 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.662429 (+ 246us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.662838 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1312,"mutex_wait_us":253,"thread_start_us":146,"threads_started":1}
I20260807 18:38:38.664116 18273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.648221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.6.138.66:35267)
0807 18:38:38.648957 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.648971 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.649099 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.649493 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.649499 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.649795 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.650445 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.650467 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.651744 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.651750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.653402 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.653417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.654332 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.654335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.654443 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.655256 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.655288 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.657479 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.661151 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.661154 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.661157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.661408 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.661805 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.661807 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.661809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.661886 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.662379 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.662385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.662492 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.663391 (+ 899us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.663762 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":217,"spinlock_wait_cycles":33408,"thread_start_us":103,"threads_started":1}
I20260807 18:38:38.665095 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.645661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:54311 (local address 127.6.138.124:38831)
0807 18:38:38.645803 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.645808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.647073 (+ 1265us) server_negotiation.cc:407] Connection header received
0807 18:38:38.648447 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.648450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.648514 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.648578 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.650632 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.651503 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.654313 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.654681 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.654786 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.658076 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.658082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.658086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.658123 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.659998 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.660735 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.660742 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.660746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.660838 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.661331 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.661337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.661338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.661606 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.661718 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.664774 (+ 3056us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.664926 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:38:38.664116 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.647744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.66:56823)
0807 18:38:38.648134 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.648391 (+ 257us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.648495 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.648984 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.648986 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.649616 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.650443 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.650465 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.651708 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.651712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.653285 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.653296 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.654153 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.654157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.654392 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.655207 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.655224 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.659160 (+ 3936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.661641 (+ 2481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.661647 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.661648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.661885 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.662211 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.662215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.662217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.662306 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.662777 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.662783 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.662864 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.663543 (+ 679us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.663705 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":350,"spinlock_wait_cycles":13696,"thread_start_us":144,"threads_started":1}
I20260807 18:38:38.666009 18269 heartbeater.cc:347] Connected to a master server at 127.6.138.125:35473
I20260807 18:38:38.666286 18269 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:38.666558 18267 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38925
I20260807 18:38:38.666646 18267 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:38.666781 18266 heartbeater.cc:347] Connected to a master server at 127.6.138.124:38831
I20260807 18:38:38.666859 18266 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:38.666949 18267 heartbeater.cc:511] Master 127.6.138.126:38925 requested a full tablet report, sending...
I20260807 18:38:38.667016 18266 heartbeater.cc:511] Master 127.6.138.124:38831 requested a full tablet report, sending...
I20260807 18:38:38.667825 18269 heartbeater.cc:511] Master 127.6.138.125:35473 requested a full tablet report, sending...
I20260807 18:38:38.668206 17813 ts_manager.cc:194] Registered new tserver with Master: 59751f6a7203483c9bdb4c47feda63ab (127.6.138.66:44067)
I20260807 18:38:38.668650 16554 ts_manager.cc:194] Re-registered known tserver with Master: 59751f6a7203483c9bdb4c47feda63ab (127.6.138.66:44067)
I20260807 18:38:38.668666 17519 ts_manager.cc:194] Registered new tserver with Master: 59751f6a7203483c9bdb4c47feda63ab (127.6.138.66:44067)
I20260807 18:38:38.669366 16554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:56823
I20260807 18:38:38.670038 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.658028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.0.0.1:59258)
0807 18:38:38.658341 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.658355 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.659545 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.664653 (+ 5108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.664656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.664676 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:38.664786 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.664793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.666836 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.666843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.667930 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.667942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.668926 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.668930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.669692 (+ 762us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.669717 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.669819 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.669902 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":648}
I20260807 18:38:38.670051 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.658744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.6.138.124:38831)
0807 18:38:38.664349 (+ 5605us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.664355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.664371 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:38.664414 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.664417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.664474 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.664906 (+ 432us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:38.665029 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.666712 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.668079 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.668786 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.669037 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.669195 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.669837 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.669915 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5464,"thread_start_us":97,"threads_started":1}
I20260807 18:38:38.670647 17828 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 18:38:38.671051 17872 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aecb998e302b4c218cfdbe152b4da9b1. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.671183 17872 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: This master's current role is: LEARNER
W20260807 18:38:38.782773 18275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:38.783084 18275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:38.783130 18275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:38.787348 18275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:38.787420 18275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:38.787523 18275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:38.793080 18275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab
--principal=oryx/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:43671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=41225
--tserver_master_addrs=127.6.138.126:38925,127.6.138.125:35473,127.6.138.124:38831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:38.794394 18275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:38.795595 18275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:38.806479 18286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:38.806454 18284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:38.806500 18283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:38.806833 18275 server_base.cc:1034] running on GCE node
I20260807 18:38:38.807320 18275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:38.807994 18275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:38.809244 18275 hybrid_clock.cc:648] HybridClock initialized: now 1786127918809218 us; error 80 us; skew 500 ppm
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:38.812566 18275 init.cc:377] Logged in from keytab as oryx/127.6.138.67@KRBTEST.COM (short username oryx)
I20260807 18:38:38.813585 18275 webserver.cc:460] Webserver started at http://127.6.138.67:41225/ using document root <none> and password file <none>
I20260807 18:38:38.814213 18275 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:38.814332 18275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:38.818464 18275 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:38:38.820816 18297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.822204 18275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:38.822355 18275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "4642c706185948dc8d65e6709d4ddff4"
format_stamp: "Formatted at 2026-08-07 18:38:37 on dist-test-slave-2d19"
server_key: "811db10c34da0f6c95a94c9c2d337afe"
server_key_iv: "e0b5f3760ba6f130f75842d1fe0404c2"
server_key_version: "encryptionkey@0"
I20260807 18:38:38.822940 18275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:38.840361 18275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:38.843420 18275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:38.843643 18275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:38.844343 18275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:38.845242 18275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:38.845316 18275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.845386 18275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:38.845438 18275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:38.855624 18275 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:43671
I20260807 18:38:38.855652 18462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:43671 every 8 connection(s)
I20260807 18:38:38.856801 18275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:38.865285 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 18275
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.124@KRBTEST.COM
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.125@KRBTEST.COM
Aug 07 18:38:38 dist-test-slave-2d19 krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127918, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:38.875082 18470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.860498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.6.138.67:34123)
0807 18:38:38.860730 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.860744 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.861152 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.861669 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.861677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.862178 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.862695 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.862702 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.863974 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.863980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.865127 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.865141 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.865865 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.865869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.865961 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.866467 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.866487 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.868988 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.871141 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.871151 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.871172 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.871575 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.871960 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.871963 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.871965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.872180 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.872804 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.872812 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.873177 (+ 365us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.873393 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.873809 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":154,"spinlock_wait_cycles":17920,"thread_start_us":86,"threads_started":1}
I20260807 18:38:38.875710 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.858833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:59633 (local address 127.6.138.126:38925)
0807 18:38:38.859038 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.859043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.860115 (+ 1072us) server_negotiation.cc:407] Connection header received
0807 18:38:38.861326 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.861330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.861390 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.861446 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.862857 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.863936 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.867738 (+ 3802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.868139 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.868291 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.871373 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.871377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.871380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.871411 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.873193 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.873695 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.873698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.873700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.874216 (+ 516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.874531 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.874534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.874536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.874750 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.874843 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.875425 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.875553 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:38:38.875939 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.860625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:34123 (local address 127.6.138.125:35473)
0807 18:38:38.860752 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.860757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.860808 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:38:38.861327 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.861330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.861390 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.861446 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.862857 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.863785 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.865282 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.865604 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.865689 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.869213 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.869217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.869221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.869254 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.870996 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.871709 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.871714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.871717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.871818 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.872324 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.872328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.872331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.872616 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.872705 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.875662 (+ 2957us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.875784 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:38:38.876605 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.859340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.67:59633)
0807 18:38:38.859903 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.860034 (+ 131us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.861152 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.861669 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.861677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.862178 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.862681 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.862697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.864071 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.864074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.867614 (+ 3540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.867626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.868444 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.868448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.868558 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.869127 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.869147 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.871208 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.873328 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.873331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.873334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.873580 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.874333 (+ 753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.874336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.874338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.874427 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.874887 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.874890 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.874991 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.875085 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.875228 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":192,"spinlock_wait_cycles":24448,"thread_start_us":138,"threads_started":1}
I20260807 18:38:38.877163 18465 heartbeater.cc:347] Connected to a master server at 127.6.138.125:35473
I20260807 18:38:38.877424 18465 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:38.878113 18465 heartbeater.cc:511] Master 127.6.138.125:35473 requested a full tablet report, sending...
I20260807 18:38:38.879473 18464 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38925
I20260807 18:38:38.879573 18464 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:38.879776 18464 heartbeater.cc:511] Master 127.6.138.126:38925 requested a full tablet report, sending...
I20260807 18:38:38.880527 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.858833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:60155 (local address 127.6.138.124:38831)
0807 18:38:38.859154 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.859158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.860074 (+ 916us) server_negotiation.cc:407] Connection header received
0807 18:38:38.861286 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.861290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.861353 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.861463 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.862843 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.863389 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.864706 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.864934 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.865065 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.876500 (+ 11435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.876504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.876507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.876537 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.878439 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.878990 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.878994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.878996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.879092 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.879421 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.879425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.879428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.879676 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.879775 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.880157 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.880352 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260807 18:38:38.880538 17519 ts_manager.cc:194] Registered new tserver with Master: 4642c706185948dc8d65e6709d4ddff4 (127.6.138.67:43671)
I20260807 18:38:38.881064 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.859340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.6.138.67:60155)
0807 18:38:38.859903 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.859959 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.861053 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.861668 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.861675 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.862126 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:38.862678 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.862693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.863555 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.863559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.864566 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.864576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.865108 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.865113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.865309 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.866008 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:38.866026 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:38.876302 (+ 10276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:38.878578 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.878584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.878589 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.878849 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.879212 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:38.879214 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:38.879216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:38.879310 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:38.879840 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:38.879843 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:38.879941 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.880688 (+ 747us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.880848 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":52,"spinlock_wait_cycles":96640,"thread_start_us":221,"threads_started":1}
I20260807 18:38:38.881474 16554 ts_manager.cc:194] Re-registered known tserver with Master: 4642c706185948dc8d65e6709d4ddff4 (127.6.138.67:43671)
I20260807 18:38:38.881702 18463 heartbeater.cc:347] Connected to a master server at 127.6.138.124:38831
I20260807 18:38:38.881798 18463 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:38.881961 18463 heartbeater.cc:511] Master 127.6.138.124:38831 requested a full tablet report, sending...
I20260807 18:38:38.882189 16554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:59633
I20260807 18:38:38.882709 17813 ts_manager.cc:194] Registered new tserver with Master: 4642c706185948dc8d65e6709d4ddff4 (127.6.138.67:43671)
I20260807 18:38:38.882786 18480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.868058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32942 (local address 127.6.138.126:38925)
0807 18:38:38.868453 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.868458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.870011 (+ 1553us) server_negotiation.cc:407] Connection header received
0807 18:38:38.870153 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.870156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.870214 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.870306 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.871113 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.871880 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.872998 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.873365 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.873437 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.877377 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.877381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.877383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.877411 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.879207 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.879840 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.879844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.879845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.879934 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.880367 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.880370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.880372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.880885 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.881012 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.882484 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.882635 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":110,"threads_started":1}
I20260807 18:38:38.885010 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.868305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.6.138.124:38831)
0807 18:38:38.868442 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.868446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.868865 (+ 419us) server_negotiation.cc:407] Connection header received
0807 18:38:38.869074 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.869078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.869130 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.869192 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.870746 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.871531 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.872904 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.873239 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.873314 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.877810 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.877814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.877817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.877845 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.879757 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.880288 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.880292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.880294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.880385 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.880725 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.880730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.880732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.880953 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.881065 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.884630 (+ 3565us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.884857 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:38:38.890517 16554 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:38.895756 16554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:38.899195 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.868993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44342 (local address 127.6.138.125:35473)
0807 18:38:38.869696 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.869701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.873920 (+ 4219us) server_negotiation.cc:407] Connection header received
0807 18:38:38.874050 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.874053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.874119 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.874188 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:38.875066 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.881616 (+ 6550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.889270 (+ 7654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.889636 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.889753 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.894593 (+ 4840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:38.894599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:38.894604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:38.894641 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:38.896902 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.897583 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.897587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.897589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.897694 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:38.898080 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:38.898086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:38.898091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:38.898329 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:38.898433 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.898838 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.899017 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":171,"threads_started":1}
I20260807 18:38:38.899461 17760 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: NON_VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: true } } }
I20260807 18:38:38.899617 17760 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:38.903688 18493 raft_consensus.cc:1050] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: attempting to promote NON_VOTER a9b8b0ac167c4d338f13c995340ab883 to VOTER
I20260807 18:38:38.904050 18493 consensus_queue.cc:237] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } }
I20260807 18:38:38.907756 17828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 LEARNER]: Refusing update from remote peer aecb998e302b4c218cfdbe152b4da9b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 18:38:38.908164 17532 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Refusing update from remote peer aecb998e302b4c218cfdbe152b4da9b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 18:38:38.908186 17760 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 18:38:38.908600 18487 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 18:38:38.910539 17760 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a9b8b0ac167c4d338f13c995340ab883 (127.6.138.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:38.910975 17532 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a9b8b0ac167c4d338f13c995340ab883 (127.6.138.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:38.911585 17828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a9b8b0ac167c4d338f13c995340ab883 (127.6.138.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:38.912492 18487 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:38.912515 17760 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:38.912648 18487 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:38.912655 17760 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:38.912812 17760 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:38.912889 17760 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:38.913673 17872 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:38.913796 17872 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:38:38.914531 18499 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aecb998e302b4c218cfdbe152b4da9b1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:38.914667 18499 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:38:38.921362 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.915028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43671 (local address 127.0.0.1:59840)
0807 18:38:38.915205 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.915223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.915319 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.916667 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.916670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.916687 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:38.916811 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.916817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.918943 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.918946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.920002 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.920011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.920985 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.920988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.921072 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.921099 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.921150 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.921219 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 18:38:38.921510 18503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.915702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:43745 (local address 127.0.0.1:37086)
0807 18:38:38.915983 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.915997 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.916082 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.916982 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.916987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.917008 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:38.917127 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.917135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.918583 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.918589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.919611 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.919622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.921012 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.921016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.921120 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.921144 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.921219 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.921300 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":120,"threads_started":1}
I20260807 18:38:38.921528 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.916148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.6.138.65:43745)
0807 18:38:38.916473 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.916490 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.916516 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:38:38.916600 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.916608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.916818 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.917009 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:38.917294 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.918452 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.919772 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.920874 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.921008 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.921188 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.921261 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.921368 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":134,"threads_started":1}
I20260807 18:38:38.922194 18500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.915331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:44067 (local address 127.0.0.1:51816)
0807 18:38:38.915617 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:38.915631 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:38.915717 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:38.917864 (+ 2147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:38.917867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:38.917890 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:38.917985 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.917991 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.919368 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.919372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.920656 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:38.920670 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.921790 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.921795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:38.921879 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.921898 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:38:38.921968 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 18:38:38.922022 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":127,"threads_started":1}
I20260807 18:38:38.922271 18506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.916215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51816 (local address 127.6.138.66:44067)
0807 18:38:38.917321 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.917333 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.917357 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:38:38.917440 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.917449 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.917648 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.917856 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:38.918118 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.919199 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.920820 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.921643 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.921760 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.921896 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.922010 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.922126 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":990,"thread_start_us":131,"threads_started":1}
I20260807 18:38:38.922905 18501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:38.915307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59840 (local address 127.6.138.67:43671)
0807 18:38:38.915991 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:38.916006 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:38.916037 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:38:38.916133 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:38.916141 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:38.916496 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:38.917535 (+ 1039us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:38.917683 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.918811 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.920164 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:38.920848 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:38.922152 (+ 1304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:38.922638 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:38.922682 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 18:38:38.922754 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":149,"threads_started":1}
I20260807 18:38:38.925437 18367 tablet_service.cc:1467] Processing CreateTablet for tablet d3cd205685c24612b95760e86661b064 (DEFAULT_TABLE table=test-table [id=7a924cacdbab4293a95e3af3b47a53bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:38.925529 18185 tablet_service.cc:1467] Processing CreateTablet for tablet d3cd205685c24612b95760e86661b064 (DEFAULT_TABLE table=test-table [id=7a924cacdbab4293a95e3af3b47a53bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:38.925499 17977 tablet_service.cc:1467] Processing CreateTablet for tablet d3cd205685c24612b95760e86661b064 (DEFAULT_TABLE table=test-table [id=7a924cacdbab4293a95e3af3b47a53bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:38.926584 18367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3cd205685c24612b95760e86661b064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:38.926642 17977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3cd205685c24612b95760e86661b064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:38.926642 18185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3cd205685c24612b95760e86661b064. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:38.934370 18507 tablet_bootstrap.cc:492] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: Bootstrap starting.
I20260807 18:38:38.935395 18508 tablet_bootstrap.cc:492] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: Bootstrap starting.
I20260807 18:38:38.936558 18512 tablet_bootstrap.cc:492] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: Bootstrap starting.
I20260807 18:38:38.937458 18508 tablet_bootstrap.cc:654] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:38.937469 18507 tablet_bootstrap.cc:654] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:38.938397 18508 log.cc:824] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:38.938671 18512 tablet_bootstrap.cc:654] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:38.939116 18507 log.cc:824] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:38.939469 18512 log.cc:824] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:38.941473 18507 tablet_bootstrap.cc:492] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: No bootstrap required, opened a new log
I20260807 18:38:38.941493 18508 tablet_bootstrap.cc:492] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: No bootstrap required, opened a new log
I20260807 18:38:38.941543 18512 tablet_bootstrap.cc:492] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: No bootstrap required, opened a new log
I20260807 18:38:38.941723 18512 ts_tablet_manager.cc:1397] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:38:38.941723 18508 ts_tablet_manager.cc:1397] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:38:38.941726 18507 ts_tablet_manager.cc:1397] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 18:38:38.946200 18507 raft_consensus.cc:348] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:38.946357 18508 raft_consensus.cc:348] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:38.946357 18512 raft_consensus.cc:348] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:38.946558 18507 raft_consensus.cc:374] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:38.946619 18507 raft_consensus.cc:729] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4642c706185948dc8d65e6709d4ddff4, State: Initialized, Role: FOLLOWER
I20260807 18:38:38.946632 18508 raft_consensus.cc:374] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:38.946632 18512 raft_consensus.cc:374] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:38.946687 18512 raft_consensus.cc:729] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59751f6a7203483c9bdb4c47feda63ab, State: Initialized, Role: FOLLOWER
I20260807 18:38:38.946687 18508 raft_consensus.cc:729] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52badaa1b1204ab1b901c2d11e7e1a79, State: Initialized, Role: FOLLOWER
I20260807 18:38:38.947157 18507 consensus_queue.cc:260] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:38.947178 18512 consensus_queue.cc:260] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:38.947178 18508 consensus_queue.cc:260] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:38.947974 18076 heartbeater.cc:503] Master 127.6.138.126:38925 was elected leader, sending a full tablet report...
I20260807 18:38:38.948377 18507 ts_tablet_manager.cc:1428] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 18:38:38.949690 18267 heartbeater.cc:503] Master 127.6.138.126:38925 was elected leader, sending a full tablet report...
I20260807 18:38:38.949887 18464 heartbeater.cc:503] Master 127.6.138.126:38925 was elected leader, sending a full tablet report...
I20260807 18:38:38.952183 18512 ts_tablet_manager.cc:1428] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260807 18:38:38.952855 18508 ts_tablet_manager.cc:1428] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260807 18:38:39.049888 18520 raft_consensus.cc:482] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:39.050225 18520 raft_consensus.cc:504] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:39.051553 18520 leader_election.cc:288] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4642c706185948dc8d65e6709d4ddff4 (127.6.138.67:43671), 52badaa1b1204ab1b901c2d11e7e1a79 (127.6.138.65:43745)
I20260807 18:38:39.055845 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:39.052357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:35881 (local address 127.6.138.65:43745)
0807 18:38:39.052534 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:39.052540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:39.052556 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:39.052621 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:39.052623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:39.052661 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:39.052756 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:39.053025 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.053526 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.054687 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.055189 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.055325 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:39.055380 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:39.055629 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 18:38:39.055703 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:38:39.055845 18268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:39.052203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:43745 (local address 127.6.138.66:35881)
0807 18:38:39.052330 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:39.052343 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:39.052436 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:39.052770 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:39.052773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:39.052789 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:39.052905 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:39.052910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.053723 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.053726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:39.054550 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:39.054560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.055318 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.055321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:39.055434 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:39.055515 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 18:38:39.055608 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:38:39.055702 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":29}
I20260807 18:38:39.055852 18501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:39.052037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:47389 (local address 127.6.138.67:43671)
0807 18:38:39.052200 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:39.052205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:39.052224 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:38:39.052319 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:39.052323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:39.052376 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:39.052532 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:39.052834 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.053386 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.054680 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.055218 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.055357 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:39.055409 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:39.055651 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:38:39.055728 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:38:39.055852 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:39.051833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43671 (local address 127.6.138.66:47389)
0807 18:38:39.052012 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:39.052030 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:39.052143 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:39.052567 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:39.052570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:39.052589 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:39.052712 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:39.052717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.053519 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.053523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:39.054548 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:39.054559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.055352 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.055356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:39.055519 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:39.055538 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:38:39.055632 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 18:38:39.055711 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":57}
I20260807 18:38:39.056725 18010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cd205685c24612b95760e86661b064" candidate_uuid: "59751f6a7203483c9bdb4c47feda63ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" is_pre_election: true
I20260807 18:38:39.056723 18391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cd205685c24612b95760e86661b064" candidate_uuid: "59751f6a7203483c9bdb4c47feda63ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4642c706185948dc8d65e6709d4ddff4" is_pre_election: true
I20260807 18:38:39.057070 18391 raft_consensus.cc:2454] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59751f6a7203483c9bdb4c47feda63ab in term 0.
I20260807 18:38:39.057076 18010 raft_consensus.cc:2454] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59751f6a7203483c9bdb4c47feda63ab in term 0.
I20260807 18:38:39.057794 18115 leader_election.cc:302] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52badaa1b1204ab1b901c2d11e7e1a79, 59751f6a7203483c9bdb4c47feda63ab; no voters:
I20260807 18:38:39.058355 18520 raft_consensus.cc:2781] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:39.058450 18520 raft_consensus.cc:482] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:39.058535 18520 raft_consensus.cc:3037] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:39.059782 18520 raft_consensus.cc:504] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:39.060169 18520 leader_election.cc:288] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [CANDIDATE]: Term 1 election: Requested vote from peers 4642c706185948dc8d65e6709d4ddff4 (127.6.138.67:43671), 52badaa1b1204ab1b901c2d11e7e1a79 (127.6.138.65:43745)
I20260807 18:38:39.060738 18391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cd205685c24612b95760e86661b064" candidate_uuid: "59751f6a7203483c9bdb4c47feda63ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4642c706185948dc8d65e6709d4ddff4"
I20260807 18:38:39.060789 18010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cd205685c24612b95760e86661b064" candidate_uuid: "59751f6a7203483c9bdb4c47feda63ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52badaa1b1204ab1b901c2d11e7e1a79"
I20260807 18:38:39.060894 18391 raft_consensus.cc:3037] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:39.060904 18010 raft_consensus.cc:3037] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:39.062288 18391 raft_consensus.cc:2454] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59751f6a7203483c9bdb4c47feda63ab in term 1.
I20260807 18:38:39.062288 18010 raft_consensus.cc:2454] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59751f6a7203483c9bdb4c47feda63ab in term 1.
I20260807 18:38:39.062592 18115 leader_election.cc:302] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52badaa1b1204ab1b901c2d11e7e1a79, 59751f6a7203483c9bdb4c47feda63ab; no voters:
I20260807 18:38:39.063004 18520 raft_consensus.cc:2781] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:39.063292 18520 raft_consensus.cc:686] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 1 LEADER]: Becoming Leader. State: Replica: 59751f6a7203483c9bdb4c47feda63ab, State: Running, Role: LEADER
I20260807 18:38:39.063733 18520 consensus_queue.cc:237] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:39.067680 16554 catalog_manager.cc:5495] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 59751f6a7203483c9bdb4c47feda63ab (127.6.138.66). New cstate: current_term: 1 leader_uuid: "59751f6a7203483c9bdb4c47feda63ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } health_report { overall_health: UNKNOWN } } }
W20260807 18:38:39.108510 18466 tablet.cc:2369] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:39.117581 18506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:39.113479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.6.138.66:44067)
0807 18:38:39.113621 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:39.113625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:39.113639 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:38:39.113739 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:39.113741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:39.113797 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:39.113893 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:39.114198 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.114771 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.115746 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.116032 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.116125 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:39.116484 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:39.117050 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:39.117172 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:39.117302 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 18:38:39.117363 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:38:39.124122 18010 raft_consensus.cc:1261] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 1 FOLLOWER]: Refusing update from remote peer 59751f6a7203483c9bdb4c47feda63ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:39.124159 18391 raft_consensus.cc:1261] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [term 1 FOLLOWER]: Refusing update from remote peer 59751f6a7203483c9bdb4c47feda63ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:39.125036 18522 consensus_queue.cc:1035] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:39.125378 18520 consensus_queue.cc:1035] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:39.135166 18528 mvcc.cc:205] Tried to move back new op lower bound from 7315979956724531200 to 7315979956488257536. Current Snapshot: MvccSnapshot[applied={T|T < 7315979956724531200}]
I20260807 18:38:39.142993 18527 mvcc.cc:205] Tried to move back new op lower bound from 7315979956724531200 to 7315979956488257536. Current Snapshot: MvccSnapshot[applied={T|T < 7315979956724531200}]
I20260807 18:38:39.146806 16554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:39.146996 16554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:39.152041 16554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aecb998e302b4c218cfdbe152b4da9b1: Sending DeleteTablet for 3 replicas of tablet d3cd205685c24612b95760e86661b064
I20260807 18:38:39.153168 18185 tablet_service.cc:1514] Processing DeleteTablet for tablet d3cd205685c24612b95760e86661b064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:39 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:51816
I20260807 18:38:39.153774 18536 tablet_replica.cc:330] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: stopping tablet replica
I20260807 18:38:39.154192 18536 raft_consensus.cc:2229] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:39.154631 18536 raft_consensus.cc:2258] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:39.155817 18367 tablet_service.cc:1514] Processing DeleteTablet for tablet d3cd205685c24612b95760e86661b064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:39 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:59840
I20260807 18:38:39.155911 17977 tablet_service.cc:1514] Processing DeleteTablet for tablet d3cd205685c24612b95760e86661b064 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:39 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:37086
I20260807 18:38:39.156394 18536 ts_tablet_manager.cc:1905] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:39.157302 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 16500
I20260807 18:38:39.174103 18536 ts_tablet_manager.cc:1918] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:39.174263 18536 log.cc:1196] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/d3cd205685c24612b95760e86661b064
I20260807 18:38:39.174692 18536 ts_tablet_manager.cc:1939] T d3cd205685c24612b95760e86661b064 P 59751f6a7203483c9bdb4c47feda63ab: Deleting consensus metadata
I20260807 18:38:39.175105 18539 tablet_replica.cc:330] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: stopping tablet replica
I20260807 18:38:39.175611 18539 raft_consensus.cc:2229] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:39.176075 18539 raft_consensus.cc:2258] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:39.178592 18539 ts_tablet_manager.cc:1905] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:39.180580 18540 tablet_replica.cc:330] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: stopping tablet replica
I20260807 18:38:39.180992 18540 raft_consensus.cc:2229] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:39.181401 18540 raft_consensus.cc:2258] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 18:38:39.182588 18114 connection.cc:535] server connection from 127.0.0.1:51816 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:38:39.183544 18540 ts_tablet_manager.cc:1905] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:39.186875 18540 ts_tablet_manager.cc:1918] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:39.186997 18540 log.cc:1196] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/d3cd205685c24612b95760e86661b064
I20260807 18:38:39.187368 18540 ts_tablet_manager.cc:1939] T d3cd205685c24612b95760e86661b064 P 52badaa1b1204ab1b901c2d11e7e1a79: Deleting consensus metadata
I20260807 18:38:39.188443 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:39.155086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44358 (local address 127.6.138.125:35473)
0807 18:38:39.155227 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:39.155232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:39.155249 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:38:39.155291 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:39.155294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:39.155344 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:39.155440 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:39.157314 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.158119 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.172990 (+ 14871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.173398 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.173510 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:39.183493 (+ 9983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:39.183498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:39.183501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:39.183539 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:39.185566 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:39.186140 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:39.186144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:39.186146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:39.186238 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:39.186563 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:39.186567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:39.186568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:39.186805 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:39.186932 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:39.188083 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0807 18:38:39.188232 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 18:38:39.188998 17910 connection.cc:441] server connection from 127.0.0.1:37086 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37086 (request call id 1) could send its response
I20260807 18:38:39.189489 18539 ts_tablet_manager.cc:1918] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:39.189638 18539 log.cc:1196] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/d3cd205685c24612b95760e86661b064
I20260807 18:38:39.189987 18539 ts_tablet_manager.cc:1939] T d3cd205685c24612b95760e86661b064 P 4642c706185948dc8d65e6709d4ddff4: Deleting consensus metadata
W20260807 18:38:39.191296 18302 connection.cc:441] server connection from 127.0.0.1:59840 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59840 (request call id 1) could send its response
I20260807 18:38:39.198458 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:39.155219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.6.138.124:38831)
0807 18:38:39.155335 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:39.155340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:39.187828 (+ 32488us) server_negotiation.cc:407] Connection header received
0807 18:38:39.188057 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:39.188061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:39.188131 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:39.188218 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:39.189190 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.189983 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.191122 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.191460 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.191685 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:39.193797 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:39.193800 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:39.193803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:39.193832 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:39.195878 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:39.196573 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:39.196576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:39.196579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:39.196670 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:39.197365 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:39.197369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:39.197370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:39.197642 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:39.197774 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:39.198128 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 18:38:39.198275 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:38:39.202256 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:39.191818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.6.138.124:38831)
0807 18:38:39.191951 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:39.191955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:39.192201 (+ 246us) server_negotiation.cc:407] Connection header received
0807 18:38:39.192422 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:39.192426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:39.192475 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:39.192536 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:39.193471 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.194280 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.195768 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.196118 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.196214 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:39.197923 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:39.197927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:39.197929 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:39.197954 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:39.200367 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:39.200879 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:39.200882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:39.200884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:39.200968 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:39.201266 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:39.201270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:39.201271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:39.201446 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:39.201592 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:39.201836 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 18:38:39.201968 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:38:39.204700 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:39.192974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.6.138.125:35473)
0807 18:38:39.193099 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:39.193103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:39.194091 (+ 988us) server_negotiation.cc:407] Connection header received
0807 18:38:39.194301 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:39.194305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:39.194392 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:39.194449 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:39.196238 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.197109 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.198258 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:39.198636 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:39.198719 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:39.200373 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:39.200377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:39.200384 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:39.200407 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:39.202447 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:39.203198 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:39.203202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:39.203204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:39.203319 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:39.203698 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:39.203702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:39.203703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:39.203900 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:39.204143 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:39.204424 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:38:39.204543 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:38:40.139309 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.138723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.65:55301)
0807 18:38:40.139031 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.139134 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":132,"threads_started":1}
W20260807 18:38:40.139757 18076 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:38925 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:40.143306 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.142776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.66:32995)
0807 18:38:40.143075 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.143196 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":113,"threads_started":1}
W20260807 18:38:40.143980 18267 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:38925 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:40.145349 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.144937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.67:45405)
0807 18:38:40.145197 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.145269 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":68,"threads_started":1}
W20260807 18:38:40.145653 18464 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:38925 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:40.652717 18553 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aecb998e302b4c218cfdbe152b4da9b1)
I20260807 18:38:40.652930 18553 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } }
I20260807 18:38:40.653271 18554 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aecb998e302b4c218cfdbe152b4da9b1)
I20260807 18:38:40.653507 18554 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } }
I20260807 18:38:40.654297 18553 leader_election.cc:288] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925), ab5e1635aad8494e88708702bf15dab4 (127.6.138.125:35473)
I20260807 18:38:40.655112 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.654612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33416)
0807 18:38:40.654877 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.654995 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":57,"threads_started":1}
I20260807 18:38:40.655254 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.654441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33406)
0807 18:38:40.655080 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.655163 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":69,"threads_started":1}
W20260807 18:38:40.655439 17796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
W20260807 18:38:40.655628 17491 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:40.655965 18554 leader_election.cc:288] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925), a9b8b0ac167c4d338f13c995340ab883 (127.6.138.124:38831)
I20260807 18:38:40.656561 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.656163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33418)
0807 18:38:40.656409 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.656474 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
W20260807 18:38:40.656798 17796 leader_election.cc:334] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925): Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:40.656883 18561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.656538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33432)
0807 18:38:40.656730 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.656793 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260807 18:38:40.656996 17491 leader_election.cc:334] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925): Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:40.660028 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.655102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38831 (local address 127.0.0.1:59280)
0807 18:38:40.655284 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.655301 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:40.655419 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:40.656172 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:40.656178 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:40.656203 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:40.656364 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:40.656371 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:40.657639 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:40.657642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:40.658766 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:40.658777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:40.659508 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:40.659512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:40.659676 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:40.659699 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:38:40.659799 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:38:40.659881 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":24,"thread_start_us":67,"threads_started":1}
I20260807 18:38:40.660024 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.655476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59280 (local address 127.6.138.124:38831)
0807 18:38:40.655773 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:40.655778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:40.655794 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:40.655844 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:40.655848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:40.655896 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:40.656002 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:40.656511 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:40.657534 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:40.658902 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:40.659359 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:40.659524 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:40.659601 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:40.659819 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 18:38:40.659899 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":227,"thread_start_us":71,"threads_started":1}
I20260807 18:38:40.660338 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.654920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.6.138.125:35473)
0807 18:38:40.656178 (+ 1258us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:40.656184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:40.656200 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:40.656274 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:40.656279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:40.656329 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:40.656842 (+ 513us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:40.656945 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:40.657805 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:40.659190 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:40.659706 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:40.659835 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:40.659896 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:40.660104 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 18:38:40.660179 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":1179,"thread_start_us":93,"threads_started":1}
I20260807 18:38:40.660351 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.654822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:35473 (local address 127.0.0.1:44364)
0807 18:38:40.655419 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.655435 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:40.655618 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:40.656539 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:40.656543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:40.656566 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:40.656734 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:40.656742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:40.657919 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:40.657923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:40.659060 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:40.659071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:40.659839 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:40.659844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:40.659960 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:40.659987 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:38:40.660083 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:38:40.660171 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":26,"thread_start_us":61,"threads_started":1}
I20260807 18:38:40.660986 17828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab5e1635aad8494e88708702bf15dab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a9b8b0ac167c4d338f13c995340ab883" is_pre_election: true
I20260807 18:38:40.661034 17532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9b8b0ac167c4d338f13c995340ab883" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab5e1635aad8494e88708702bf15dab4" is_pre_election: true
I20260807 18:38:40.661222 17532 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b8b0ac167c4d338f13c995340ab883 in term 1.
I20260807 18:38:40.661222 17828 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab5e1635aad8494e88708702bf15dab4 in term 1.
I20260807 18:38:40.661638 17796 leader_election.cc:302] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9b8b0ac167c4d338f13c995340ab883, ab5e1635aad8494e88708702bf15dab4; no voters: aecb998e302b4c218cfdbe152b4da9b1
I20260807 18:38:40.661703 17495 leader_election.cc:302] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9b8b0ac167c4d338f13c995340ab883, ab5e1635aad8494e88708702bf15dab4; no voters: aecb998e302b4c218cfdbe152b4da9b1
I20260807 18:38:40.661896 18553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 18:38:40.661967 18553 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aecb998e302b4c218cfdbe152b4da9b1)
I20260807 18:38:40.662014 18553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:38:40.662106 18554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 18:38:40.662204 18554 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aecb998e302b4c218cfdbe152b4da9b1)
I20260807 18:38:40.662267 18554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:38:40.663110 18553 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } }
I20260807 18:38:40.663489 18553 leader_election.cc:288] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 2 election: Requested vote from peers aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925), ab5e1635aad8494e88708702bf15dab4 (127.6.138.125:35473)
I20260807 18:38:40.663421 18554 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } }
I20260807 18:38:40.664006 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.663671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33442)
0807 18:38:40.663785 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.663934 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 18:38:40.664155 18554 leader_election.cc:288] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [CANDIDATE]: Term 2 election: Requested vote from peers aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925), a9b8b0ac167c4d338f13c995340ab883 (127.6.138.124:38831)
I20260807 18:38:40.664129 17532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9b8b0ac167c4d338f13c995340ab883" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab5e1635aad8494e88708702bf15dab4"
I20260807 18:38:40.664307 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.663984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33452)
0807 18:38:40.664146 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.664201 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 18:38:40.664333 17532 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a9b8b0ac167c4d338f13c995340ab883 in current term 2: Already voted for candidate ab5e1635aad8494e88708702bf15dab4 in this term.
I20260807 18:38:40.664549 17828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab5e1635aad8494e88708702bf15dab4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a9b8b0ac167c4d338f13c995340ab883"
I20260807 18:38:40.664732 17828 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ab5e1635aad8494e88708702bf15dab4 in current term 2: Already voted for candidate a9b8b0ac167c4d338f13c995340ab883 in this term.
I20260807 18:38:40.665266 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.665045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33466)
0807 18:38:40.665141 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.665188 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 18:38:40.665374 17796 leader_election.cc:334] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925): Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:40.665453 17796 leader_election.cc:302] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9b8b0ac167c4d338f13c995340ab883; no voters: ab5e1635aad8494e88708702bf15dab4, aecb998e302b4c218cfdbe152b4da9b1
I20260807 18:38:40.665623 18553 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 18:38:40.666318 18557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:40.666006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33472)
0807 18:38:40.666215 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:40.666265 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260807 18:38:40.666515 17491 leader_election.cc:334] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925): Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:40.666625 17491 leader_election.cc:302] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab5e1635aad8494e88708702bf15dab4; no voters: a9b8b0ac167c4d338f13c995340ab883, aecb998e302b4c218cfdbe152b4da9b1
I20260807 18:38:40.666796 18554 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 18:38:41.141511 18568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:41.140645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.65:44533)
0807 18:38:41.141250 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:41.141341 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":307,"threads_started":1}
I20260807 18:38:41.145239 18569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:41.144703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.66:51451)
0807 18:38:41.145033 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:41.145116 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":82,"threads_started":1}
I20260807 18:38:41.147068 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:41.146280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.67:55851)
0807 18:38:41.146781 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:41.146897 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":186,"threads_started":1}
I20260807 18:38:42.143077 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.142517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.65:45933)
0807 18:38:42.142859 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.142943 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":115,"threads_started":1}
I20260807 18:38:42.146694 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.146085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.66:34909)
0807 18:38:42.146480 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.146582 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":82,"threads_started":1}
I20260807 18:38:42.148378 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.147888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.67:40989)
0807 18:38:42.148164 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.148234 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":92,"threads_started":1}
I20260807 18:38:42.500355 18575 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:42.500510 18575 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } }
I20260807 18:38:42.500973 18575 leader_election.cc:288] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925), ab5e1635aad8494e88708702bf15dab4 (127.6.138.125:35473)
I20260807 18:38:42.501569 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.501169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33724)
0807 18:38:42.501402 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.501471 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":62,"threads_started":1}
I20260807 18:38:42.501742 17532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9b8b0ac167c4d338f13c995340ab883" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab5e1635aad8494e88708702bf15dab4" is_pre_election: true
I20260807 18:38:42.501910 17532 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b8b0ac167c4d338f13c995340ab883 in term 2.
I20260807 18:38:42.502270 17796 leader_election.cc:302] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b8b0ac167c4d338f13c995340ab883, ab5e1635aad8494e88708702bf15dab4; no voters:
I20260807 18:38:42.502465 18575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 18:38:42.502550 18575 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:42.502578 18575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:38:42.502872 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.502605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33740)
0807 18:38:42.502738 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.502802 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:38:42.503065 17796 leader_election.cc:334] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925): Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:42.503922 18575 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } }
I20260807 18:38:42.504386 18575 leader_election.cc:288] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 3 election: Requested vote from peers aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925), ab5e1635aad8494e88708702bf15dab4 (127.6.138.125:35473)
I20260807 18:38:42.504747 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.504540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33744)
0807 18:38:42.504649 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.504697 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 18:38:42.504940 17532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9b8b0ac167c4d338f13c995340ab883" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab5e1635aad8494e88708702bf15dab4"
I20260807 18:38:42.505074 17532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:38:42.505906 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.505699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33750)
0807 18:38:42.505806 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.505853 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 18:38:42.506080 17796 leader_election.cc:334] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925): Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
I20260807 18:38:42.506338 17532 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b8b0ac167c4d338f13c995340ab883 in term 3.
I20260807 18:38:42.506623 17796 leader_election.cc:302] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9b8b0ac167c4d338f13c995340ab883, ab5e1635aad8494e88708702bf15dab4; no voters: aecb998e302b4c218cfdbe152b4da9b1
I20260807 18:38:42.506779 18575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:38:42.507040 18575 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [term 3 LEADER]: Becoming Leader. State: Replica: a9b8b0ac167c4d338f13c995340ab883, State: Running, Role: LEADER
I20260807 18:38:42.507314 18575 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } }
I20260807 18:38:42.508904 18578 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9b8b0ac167c4d338f13c995340ab883. Latest consensus state: current_term: 3 leader_uuid: "a9b8b0ac167c4d338f13c995340ab883" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:42.509032 18578 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:42.509349 18582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:42.511708 18582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:42.512101 18582 catalog_manager.cc:1259] Loaded cluster ID: 589dd4e6265641c3bb2a26ad1e6a6187
I20260807 18:38:42.512145 18582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:42.512748 18582 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:42.513108 18582 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Loaded TSK: 0
I20260807 18:38:42.513777 18582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:38:42.521951 17812 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:42.525137 17812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:42.528414 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.528115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33766)
0807 18:38:42.528247 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.528345 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 18:38:42.528486 17532 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [term 3 FOLLOWER]: Refusing update from remote peer a9b8b0ac167c4d338f13c995340ab883: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 18:38:42.529143 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.528920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.0.0.1:33772)
0807 18:38:42.529025 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.529078 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 18:38:42.529253 17796 consensus_peers.cc:487] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 -> Peer aecb998e302b4c218cfdbe152b4da9b1 (127.6.138.126:38925): Couldn't send request to peer aecb998e302b4c218cfdbe152b4da9b1. Status: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 18:38:42.529299 18578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 18:38:42.530992 18585 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9b8b0ac167c4d338f13c995340ab883. Latest consensus state: current_term: 3 leader_uuid: "a9b8b0ac167c4d338f13c995340ab883" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:42.531133 18585 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:38:42.532370 18578 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a9b8b0ac167c4d338f13c995340ab883" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:42.532497 18578 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:42.532763 18585 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "a9b8b0ac167c4d338f13c995340ab883" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aecb998e302b4c218cfdbe152b4da9b1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38925 } } peers { permanent_uuid: "ab5e1635aad8494e88708702bf15dab4" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 35473 } attrs { promote: false } } peers { permanent_uuid: "a9b8b0ac167c4d338f13c995340ab883" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38831 } attrs { promote: false } } }
I20260807 18:38:42.532892 18585 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5e1635aad8494e88708702bf15dab4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:38:42.533020 18588 mvcc.cc:205] Tried to move back new op lower bound from 7315979970671185920 to 7315979970592628736. Current Snapshot: MvccSnapshot[applied={T|T < 7315979970671185920}]
I20260807 18:38:42.543841 18598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.539064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:43745 (local address 127.0.0.1:37092)
0807 18:38:42.539791 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.539805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:42.539915 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:42.540170 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:42.540173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:42.540194 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:42.540329 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.540339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.541404 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.541408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.542615 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.542621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.543385 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.543397 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.543491 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.543515 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:38:42.543610 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:38:42.543686 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":55,"thread_start_us":103,"threads_started":1}
I20260807 18:38:42.543840 18603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.539518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.6.138.65:43745)
0807 18:38:42.539788 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:42.539792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:42.539883 (+ 91us) server_negotiation.cc:407] Connection header received
0807 18:38:42.540001 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:42.540003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:42.540051 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:42.540148 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:42.540485 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.541296 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.542743 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.543265 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.543385 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.543463 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:42.543633 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 18:38:42.543712 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":100,"threads_started":1}
I20260807 18:38:42.544401 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.539421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.6.138.67:43671)
0807 18:38:42.539670 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:42.539677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:42.539695 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:38:42.539753 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:42.539756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:42.539809 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:42.539898 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:42.540220 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.540943 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.542759 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.543358 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.544046 (+ 688us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.544112 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:42.544148 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 18:38:42.544217 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":71,"threads_started":1}
I20260807 18:38:42.543921 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.538723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43671 (local address 127.0.0.1:59844)
0807 18:38:42.539288 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.539319 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:42.539404 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:42.539925 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:42.539928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:42.539947 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:42.540075 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.540084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.541103 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.541108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.542609 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.542620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.543511 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.543515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.543607 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.543633 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:38:42.543687 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 18:38:42.543761 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":64,"thread_start_us":102,"threads_started":1}
I20260807 18:38:42.544562 18597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.538625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.6.138.66:44067)
0807 18:38:42.540151 (+ 1526us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:42.540156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:42.540169 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:38:42.540219 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:42.540221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:42.540299 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:42.540393 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:42.540650 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.541390 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.542983 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.543616 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.543719 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.543778 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:42.544325 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 18:38:42.544416 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1427,"thread_start_us":74,"threads_started":1}
I20260807 18:38:42.544219 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.538454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:44067 (local address 127.0.0.1:51842)
0807 18:38:42.538677 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.538695 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:42.538798 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:42.540411 (+ 1613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:42.540414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:42.540432 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:42.540523 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.540528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.541485 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.541490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.542849 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.542859 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.543836 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.543839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.543926 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.543958 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:38:42.544043 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:38:42.544109 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":103}
I20260807 18:38:42.545308 17977 tablet_service.cc:1467] Processing CreateTablet for tablet cfbe4126885345c99c6ccc1cb9913919 (DEFAULT_TABLE table=test-table [id=b33b3d8e2c824e11a4ef74209bccbe00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:42.545490 18367 tablet_service.cc:1467] Processing CreateTablet for tablet cfbe4126885345c99c6ccc1cb9913919 (DEFAULT_TABLE table=test-table [id=b33b3d8e2c824e11a4ef74209bccbe00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:42.545665 17977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfbe4126885345c99c6ccc1cb9913919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:42.545746 18367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfbe4126885345c99c6ccc1cb9913919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:42.547830 18604 tablet_bootstrap.cc:492] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: Bootstrap starting.
I20260807 18:38:42.548273 18185 tablet_service.cc:1467] Processing CreateTablet for tablet cfbe4126885345c99c6ccc1cb9913919 (DEFAULT_TABLE table=test-table [id=b33b3d8e2c824e11a4ef74209bccbe00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:42.548595 18185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfbe4126885345c99c6ccc1cb9913919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:42.548637 18605 tablet_bootstrap.cc:492] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79: Bootstrap starting.
I20260807 18:38:42.549190 18604 tablet_bootstrap.cc:654] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:42.549392 18605 tablet_bootstrap.cc:654] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:42.550578 18605 tablet_bootstrap.cc:492] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79: No bootstrap required, opened a new log
I20260807 18:38:42.550644 18605 ts_tablet_manager.cc:1397] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:42.551004 18605 raft_consensus.cc:348] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:42.551092 18605 raft_consensus.cc:374] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:42.551147 18605 raft_consensus.cc:729] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52badaa1b1204ab1b901c2d11e7e1a79, State: Initialized, Role: FOLLOWER
I20260807 18:38:42.551147 18610 tablet_bootstrap.cc:492] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: Bootstrap starting.
I20260807 18:38:42.551348 18605 consensus_queue.cc:260] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:42.551777 18605 ts_tablet_manager.cc:1428] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:42.552116 18604 tablet_bootstrap.cc:492] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: No bootstrap required, opened a new log
I20260807 18:38:42.552229 18604 ts_tablet_manager.cc:1397] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:38:42.552996 18604 raft_consensus.cc:348] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:42.553107 18604 raft_consensus.cc:374] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:42.553136 18604 raft_consensus.cc:729] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4642c706185948dc8d65e6709d4ddff4, State: Initialized, Role: FOLLOWER
I20260807 18:38:42.553256 18604 consensus_queue.cc:260] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:42.553642 18604 ts_tablet_manager.cc:1428] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:42.554031 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.552713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.65:46183)
0807 18:38:42.553867 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.553935 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1063}
I20260807 18:38:42.555119 18075 heartbeater.cc:503] Master 127.6.138.124:38831 was elected leader, sending a full tablet report...
I20260807 18:38:42.555145 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.554790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.67:34441)
0807 18:38:42.554913 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.555064 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 18:38:42.556396 18610 tablet_bootstrap.cc:654] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:42.557264 18463 heartbeater.cc:503] Master 127.6.138.124:38831 was elected leader, sending a full tablet report...
I20260807 18:38:42.557744 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.557464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.67:52901)
0807 18:38:42.557617 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.557676 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 18:38:42.558077 18610 tablet_bootstrap.cc:492] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: No bootstrap required, opened a new log
I20260807 18:38:42.558198 18610 ts_tablet_manager.cc:1397] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260807 18:38:42.558692 18610 raft_consensus.cc:348] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:42.558846 18610 raft_consensus.cc:374] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:42.558897 18610 raft_consensus.cc:729] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59751f6a7203483c9bdb4c47feda63ab, State: Initialized, Role: FOLLOWER
I20260807 18:38:42.559046 18610 consensus_queue.cc:260] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:42.559484 18610 ts_tablet_manager.cc:1428] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 18:38:42.560495 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.560170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.66:45475)
0807 18:38:42.560349 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.560416 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 18:38:42.561244 18266 heartbeater.cc:503] Master 127.6.138.124:38831 was elected leader, sending a full tablet report...
I20260807 18:38:42.561748 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.561460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.66:35825)
0807 18:38:42.561632 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.561693 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 18:38:42.638032 18618 raft_consensus.cc:482] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:42.638195 18618 raft_consensus.cc:504] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:42.639590 18618 leader_election.cc:288] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59751f6a7203483c9bdb4c47feda63ab (127.6.138.66:44067), 52badaa1b1204ab1b901c2d11e7e1a79 (127.6.138.65:43745)
I20260807 18:38:42.644227 18603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.640314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:58153 (local address 127.6.138.65:43745)
0807 18:38:42.640445 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:42.640452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:42.640468 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:42.640535 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:42.640538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:42.640582 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:42.640651 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:42.641145 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.641854 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.643236 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.643641 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.643780 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.643837 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:42.644031 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 18:38:42.644111 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:38:42.644245 18622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.639841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:43745 (local address 127.6.138.67:58153)
0807 18:38:42.640316 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.640330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:42.640421 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:42.640906 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:42.640910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:42.640928 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:42.641020 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.641027 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.642010 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.642013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.643125 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.643132 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.643786 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.643790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.643892 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.643914 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:38:42.644014 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:38:42.644091 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":150,"threads_started":1}
I20260807 18:38:42.644438 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.639767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:44067 (local address 127.6.138.67:43291)
0807 18:38:42.639890 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.639907 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:42.640029 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:42.640463 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:42.640467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:42.640487 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:42.640592 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.640598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.641657 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.641660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.642995 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:42.643006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.643964 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.643968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:42.644099 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.644134 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:38:42.644207 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 18:38:42.644316 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 18:38:42.644438 18597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.640013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:43291 (local address 127.6.138.66:44067)
0807 18:38:42.640123 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:42.640128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:42.640142 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:38:42.640192 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:42.640195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:42.640237 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:42.640340 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:42.640707 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.641541 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.643131 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.643739 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.643853 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.643900 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:42.644237 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 18:38:42.644335 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:38:42.644930 18010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbe4126885345c99c6ccc1cb9913919" candidate_uuid: "4642c706185948dc8d65e6709d4ddff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" is_pre_election: true
I20260807 18:38:42.644958 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbe4126885345c99c6ccc1cb9913919" candidate_uuid: "4642c706185948dc8d65e6709d4ddff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59751f6a7203483c9bdb4c47feda63ab" is_pre_election: true
I20260807 18:38:42.645088 18010 raft_consensus.cc:2454] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4642c706185948dc8d65e6709d4ddff4 in term 0.
I20260807 18:38:42.645218 18210 raft_consensus.cc:2454] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4642c706185948dc8d65e6709d4ddff4 in term 0.
I20260807 18:38:42.645556 18300 leader_election.cc:302] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4642c706185948dc8d65e6709d4ddff4, 52badaa1b1204ab1b901c2d11e7e1a79, 59751f6a7203483c9bdb4c47feda63ab; no voters:
I20260807 18:38:42.645874 18618 raft_consensus.cc:2781] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:42.645949 18618 raft_consensus.cc:482] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:42.646078 18618 raft_consensus.cc:3037] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:42.647105 18618 raft_consensus.cc:504] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
I20260807 18:38:42.647455 18618 leader_election.cc:288] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [CANDIDATE]: Term 1 election: Requested vote from peers 59751f6a7203483c9bdb4c47feda63ab (127.6.138.66:44067), 52badaa1b1204ab1b901c2d11e7e1a79 (127.6.138.65:43745)
I20260807 18:38:42.647835 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbe4126885345c99c6ccc1cb9913919" candidate_uuid: "4642c706185948dc8d65e6709d4ddff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59751f6a7203483c9bdb4c47feda63ab"
I20260807 18:38:42.647843 18010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbe4126885345c99c6ccc1cb9913919" candidate_uuid: "4642c706185948dc8d65e6709d4ddff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52badaa1b1204ab1b901c2d11e7e1a79"
I20260807 18:38:42.647967 18210 raft_consensus.cc:3037] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:42.647987 18010 raft_consensus.cc:3037] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:42.648974 18010 raft_consensus.cc:2454] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4642c706185948dc8d65e6709d4ddff4 in term 1.
I20260807 18:38:42.649005 18210 raft_consensus.cc:2454] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4642c706185948dc8d65e6709d4ddff4 in term 1.
I20260807 18:38:42.649302 18300 leader_election.cc:302] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4642c706185948dc8d65e6709d4ddff4, 52badaa1b1204ab1b901c2d11e7e1a79; no voters:
I20260807 18:38:42.649483 18618 raft_consensus.cc:2781] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:42.649729 18618 raft_consensus.cc:686] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 1 LEADER]: Becoming Leader. State: Replica: 4642c706185948dc8d65e6709d4ddff4, State: Running, Role: LEADER
I20260807 18:38:42.650012 18618 consensus_queue.cc:237] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } }
W20260807 18:38:42.650353 18270 tablet.cc:2369] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:42.650470 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.650230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.67:58271)
0807 18:38:42.650334 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.650391 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 18:38:42.652563 17812 catalog_manager.cc:5495] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4642c706185948dc8d65e6709d4ddff4 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "4642c706185948dc8d65e6709d4ddff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4642c706185948dc8d65e6709d4ddff4" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 } health_report { overall_health: UNKNOWN } } }
W20260807 18:38:42.660395 18078 tablet.cc:2369] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:42.674345 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.670650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59856 (local address 127.6.138.67:43671)
0807 18:38:42.670820 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:42.670826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:42.670843 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:38:42.670933 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:42.670935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:42.670992 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:42.671110 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:42.671364 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.671887 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.672825 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:42.673050 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:42.673161 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:42.673459 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:42.673935 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:42.674066 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:42.674128 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:38:42.674190 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:38:42.677639 18010 raft_consensus.cc:1261] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 1 FOLLOWER]: Refusing update from remote peer 4642c706185948dc8d65e6709d4ddff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:42.677891 18210 raft_consensus.cc:1261] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [term 1 FOLLOWER]: Refusing update from remote peer 4642c706185948dc8d65e6709d4ddff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:42.678244 18618 consensus_queue.cc:1035] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52badaa1b1204ab1b901c2d11e7e1a79" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 43745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:42.678539 18625 consensus_queue.cc:1035] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59751f6a7203483c9bdb4c47feda63ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:42.680532 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.680175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.65:47299)
0807 18:38:42.680380 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.680446 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 18:38:42.682952 18574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.682701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.67:55963)
0807 18:38:42.682819 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.682877 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 18:38:42.684981 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.684620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.65:51421)
0807 18:38:42.684842 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.684898 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260807 18:38:42.685313 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.684992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.66:53143)
0807 18:38:42.685180 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.685239 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260807 18:38:42.689458 18635 mvcc.cc:205] Tried to move back new op lower bound from 7315979971281465344 to 7315979971177332736. Current Snapshot: MvccSnapshot[applied={T|T < 7315979971281465344}]
I20260807 18:38:42.692873 17810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:42.693041 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.692230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38925 (local address 127.6.138.66:49171)
0807 18:38:42.692907 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.692965 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":604}
I20260807 18:38:42.693041 17810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:42.697330 17810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9b8b0ac167c4d338f13c995340ab883: Sending DeleteTablet for 3 replicas of tablet cfbe4126885345c99c6ccc1cb9913919
I20260807 18:38:42.698235 17977 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbe4126885345c99c6ccc1cb9913919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:42 UTC) from {username='oryx', principal='oryx/127.6.138.124@KRBTEST.COM'} at 127.0.0.1:37092
I20260807 18:38:42.698599 18645 tablet_replica.cc:330] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79: stopping tablet replica
I20260807 18:38:42.698730 18185 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbe4126885345c99c6ccc1cb9913919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:42 UTC) from {username='oryx', principal='oryx/127.6.138.124@KRBTEST.COM'} at 127.0.0.1:51842
I20260807 18:38:42.698755 18645 raft_consensus.cc:2229] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:42.698894 18645 raft_consensus.cc:2258] T cfbe4126885345c99c6ccc1cb9913919 P 52badaa1b1204ab1b901c2d11e7e1a79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:42.699060 18646 tablet_replica.cc:330] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: stopping tablet replica
I20260807 18:38:42.699214 18646 raft_consensus.cc:2229] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:42.699208 18367 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbe4126885345c99c6ccc1cb9913919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:42 UTC) from {username='oryx', principal='oryx/127.6.138.124@KRBTEST.COM'} at 127.0.0.1:59844
I20260807 18:38:42.699388 18646 raft_consensus.cc:2258] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:42.698586 17974 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbe4126885345c99c6ccc1cb9913919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:42 UTC) from {username='oryx', principal='oryx/127.6.138.124@KRBTEST.COM'} at 127.0.0.1:37092
I20260807 18:38:42.699687 18647 tablet_replica.cc:330] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: stopping tablet replica
I20260807 18:38:42.699851 18647 raft_consensus.cc:2229] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:42.700152 18646 ts_tablet_manager.cc:1905] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:42.700217 18647 raft_consensus.cc:2258] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:42.700717 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 17759
I20260807 18:38:42.703809 18647 ts_tablet_manager.cc:1905] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:42.707861 18647 ts_tablet_manager.cc:1918] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:42.708024 18647 log.cc:1196] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/cfbe4126885345c99c6ccc1cb9913919
I20260807 18:38:42.708513 18647 ts_tablet_manager.cc:1939] T cfbe4126885345c99c6ccc1cb9913919 P 4642c706185948dc8d65e6709d4ddff4: Deleting consensus metadata
I20260807 18:38:42.709815 17794 catalog_manager.cc:4841] TS 4642c706185948dc8d65e6709d4ddff4 (127.6.138.67:43671): tablet cfbe4126885345c99c6ccc1cb9913919 (table test-table [id=b33b3d8e2c824e11a4ef74209bccbe00]) successfully deleted
I20260807 18:38:42.717574 18646 ts_tablet_manager.cc:1918] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:42.717689 18646 log.cc:1196] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/cfbe4126885345c99c6ccc1cb9913919
I20260807 18:38:42.718010 18646 ts_tablet_manager.cc:1939] T cfbe4126885345c99c6ccc1cb9913919 P 59751f6a7203483c9bdb4c47feda63ab: Deleting consensus metadata
I20260807 18:38:42.718869 17796 catalog_manager.cc:4841] TS 59751f6a7203483c9bdb4c47feda63ab (127.6.138.66:44067): tablet cfbe4126885345c99c6ccc1cb9913919 (table test-table [id=b33b3d8e2c824e11a4ef74209bccbe00]) successfully deleted
I20260807 18:38:42.720988 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 18090
I20260807 18:38:42.721323 18598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:42.720953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:43745 (local address 127.0.0.1:37108)
0807 18:38:42.721146 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:42.721213 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:43745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 18:38:42.729401 17794 catalog_manager.cc:4584] TS 52badaa1b1204ab1b901c2d11e7e1a79 (127.6.138.65:43745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cfbe4126885345c99c6ccc1cb9913919: Network error: Client connection negotiation failed: client connection to 127.6.138.65:43745: connect: Connection refused (error 111)
I20260807 18:38:42.740578 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 18275
I20260807 18:38:42.758153 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 17293
I20260807 18:38:42.772730 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 17457
2026-08-07T18:38:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7822 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:42 dist-test-slave-2d19 krb5kdc[18651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:42 dist-test-slave-2d19 krb5kdc[18651](info): set up 2 sockets
Aug 07 18:38:42 dist-test-slave-2d19 krb5kdc[18651](info): commencing operation
krb5kdc: starting...
W20260807 18:38:44.090034 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:44 dist-test-slave-2d19 krb5kdc[18651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.126.keytab.
Entry for principal oryx/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.126.keytab.
I20260807 18:38:44.223799 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38003
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.126.keytab
--principal=oryx/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:38003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:44.340889 18667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:44.341231 18667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:44.341303 18667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:44.345640 18667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:44.345741 18667 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:38:44.345772 18667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:44.345816 18667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:44.345870 18667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:44.345919 18667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:44.351948 18667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:38003
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.126.keytab
--principal=oryx/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:44.353237 18667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:44.354377 18667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:44.364374 18676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:44.364405 18679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:44.364441 18667 server_base.cc:1034] running on GCE node
W20260807 18:38:44.364382 18677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:44.365319 18667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:44.366618 18667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:44.367825 18667 hybrid_clock.cc:648] HybridClock initialized: now 1786127924367790 us; error 50 us; skew 500 ppm
Aug 07 18:38:44 dist-test-slave-2d19 krb5kdc[18651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127924, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:44.376595 18667 init.cc:377] Logged in from keytab as oryx/127.6.138.126@KRBTEST.COM (short username oryx)
I20260807 18:38:44.377640 18667 webserver.cc:460] Webserver started at http://127.6.138.126:46021/ using document root <none> and password file <none>
I20260807 18:38:44.378249 18667 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:44.378320 18667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:44.378553 18667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:44.380793 18667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "ea7f5ff830824d7481f27f37986e0d80"
format_stamp: "Formatted at 2026-08-07 18:38:44 on dist-test-slave-2d19"
server_key: "3663c9a16e897906004704ce951cca05"
server_key_iv: "5ff37619488d20690689c837764e3106"
server_key_version: "encryptionkey@0"
I20260807 18:38:44.381445 18667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "ea7f5ff830824d7481f27f37986e0d80"
format_stamp: "Formatted at 2026-08-07 18:38:44 on dist-test-slave-2d19"
server_key: "3663c9a16e897906004704ce951cca05"
server_key_iv: "5ff37619488d20690689c837764e3106"
server_key_version: "encryptionkey@0"
I20260807 18:38:44.385502 18667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:38:44.388357 18690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:44.389693 18667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:38:44.389847 18667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "ea7f5ff830824d7481f27f37986e0d80"
format_stamp: "Formatted at 2026-08-07 18:38:44 on dist-test-slave-2d19"
server_key: "3663c9a16e897906004704ce951cca05"
server_key_iv: "5ff37619488d20690689c837764e3106"
server_key_version: "encryptionkey@0"
I20260807 18:38:44.389971 18667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:44.436862 18667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:44.440152 18667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:44.440385 18667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:44.449113 18667 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:38003
I20260807 18:38:44.449148 18775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:38003 every 8 connection(s)
I20260807 18:38:44.450162 18667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:44.452220 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 18667
I20260807 18:38:44.452380 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:44.452714 6697 external_mini_cluster.cc:1428] Setting key 1c49e38b44a3532c2a6d2ee4bf36e02f
I20260807 18:38:44.453370 18777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:44.459512 18777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80: Bootstrap starting.
Aug 07 18:38:44 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:44.462332 18777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:44.463646 18777 log.cc:824] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:44.465690 18777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80: No bootstrap required, opened a new log
I20260807 18:38:44.467350 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:44.454182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.6.138.126:38003)
0807 18:38:44.454656 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:44.454672 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:44.454709 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:38:44.455388 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:44.455405 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:44.455742 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:44.456101 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:44.456984 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.457994 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.458865 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.459346 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.459527 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:44.462380 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:44.462387 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:44.462433 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:44.462480 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:44.464346 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.464808 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.464814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.464818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.464933 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.465272 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.465276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.465277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.465683 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:44.465891 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:44.466219 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 18:38:44.466529 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":152,"threads_started":1}
I20260807 18:38:44.469362 18777 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea7f5ff830824d7481f27f37986e0d80" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38003 } }
I20260807 18:38:44.469677 18777 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:44.469805 18777 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea7f5ff830824d7481f27f37986e0d80, State: Initialized, Role: FOLLOWER
I20260807 18:38:44.470250 18777 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea7f5ff830824d7481f27f37986e0d80" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38003 } }
I20260807 18:38:44.470377 18777 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:44.470460 18777 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:44.470564 18777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:44.471544 18777 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea7f5ff830824d7481f27f37986e0d80" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38003 } }
I20260807 18:38:44.471933 18777 leader_election.cc:302] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea7f5ff830824d7481f27f37986e0d80; no voters:
I20260807 18:38:44.472368 18777 leader_election.cc:288] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:44.472544 18785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:44.472795 18785 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [term 1 LEADER]: Becoming Leader. State: Replica: ea7f5ff830824d7481f27f37986e0d80, State: Running, Role: LEADER
I20260807 18:38:44.473155 18785 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea7f5ff830824d7481f27f37986e0d80" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38003 } }
I20260807 18:38:44.473755 18777 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:44.475059 18788 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea7f5ff830824d7481f27f37986e0d80. Latest consensus state: current_term: 1 leader_uuid: "ea7f5ff830824d7481f27f37986e0d80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea7f5ff830824d7481f27f37986e0d80" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38003 } } }
I20260807 18:38:44.475195 18788 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:44.475596 18797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:44.475955 18786 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea7f5ff830824d7481f27f37986e0d80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea7f5ff830824d7481f27f37986e0d80" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38003 } } }
I20260807 18:38:44.476231 18786 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:44.479036 18797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:44.485464 18797 catalog_manager.cc:1347] Generated new cluster ID: 4632ce87094f4fd59bf1747b6f73425f
I20260807 18:38:44.485558 18797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:44.492843 18797 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:44.493808 18797 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:44.506115 18797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80: Generated new TSK 0
I20260807 18:38:44.506835 18797 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab.
Entry for principal oryx/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab.
I20260807 18:38:44.545710 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38003
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab
--principal=oryx/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:44.669387 18813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:44.669665 18813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:44.669710 18813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:44.673672 18813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:44.673732 18813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:44.673770 18813 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:38:44.673851 18813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:44.679304 18813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab
--principal=oryx/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.138.126:38003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:44.680366 18813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:44.681368 18813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:44.693400 18823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:44.693470 18821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:44.693751 18825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:44.693949 18813 server_base.cc:1034] running on GCE node
I20260807 18:38:44.694352 18813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:44.695062 18813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:44.696295 18813 hybrid_clock.cc:648] HybridClock initialized: now 1786127924696238 us; error 33 us; skew 500 ppm
Aug 07 18:38:44 dist-test-slave-2d19 krb5kdc[18651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127924, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:44.699169 18813 init.cc:377] Logged in from keytab as oryx/127.6.138.65@KRBTEST.COM (short username oryx)
I20260807 18:38:44.700103 18813 webserver.cc:460] Webserver started at http://127.6.138.65:40817/ using document root <none> and password file <none>
I20260807 18:38:44.700762 18813 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:44.700855 18813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:44.701086 18813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:44.703183 18813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "173662d55dd946608a6e0ffd293a2bb4"
format_stamp: "Formatted at 2026-08-07 18:38:44 on dist-test-slave-2d19"
server_key: "42c3bbc5b5ef6ed6df55c3e6c1cc1d13"
server_key_iv: "1b3766844d2d92bcb000dc3b22ca9f19"
server_key_version: "encryptionkey@0"
I20260807 18:38:44.703774 18813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "173662d55dd946608a6e0ffd293a2bb4"
format_stamp: "Formatted at 2026-08-07 18:38:44 on dist-test-slave-2d19"
server_key: "42c3bbc5b5ef6ed6df55c3e6c1cc1d13"
server_key_iv: "1b3766844d2d92bcb000dc3b22ca9f19"
server_key_version: "encryptionkey@0"
I20260807 18:38:44.707615 18813 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 18:38:44.710276 18835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:44.711582 18813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:38:44.711719 18813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "173662d55dd946608a6e0ffd293a2bb4"
format_stamp: "Formatted at 2026-08-07 18:38:44 on dist-test-slave-2d19"
server_key: "42c3bbc5b5ef6ed6df55c3e6c1cc1d13"
server_key_iv: "1b3766844d2d92bcb000dc3b22ca9f19"
server_key_version: "encryptionkey@0"
I20260807 18:38:44.711831 18813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:44.723181 18813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:44.726711 18813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:44.726925 18813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:44.728549 18813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:44.728621 18813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:44.728695 18813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:44.728734 18813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:38:44 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127924, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:44.741753 18813 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:37273
I20260807 18:38:44.742519 19006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:37273 every 8 connection(s)
I20260807 18:38:44.742908 18813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:44.744470 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 18813
I20260807 18:38:44.744630 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:44.745025 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:44.729313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:57513 (local address 127.6.138.126:38003)
0807 18:38:44.729470 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:44.729475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:44.730488 (+ 1013us) server_negotiation.cc:407] Connection header received
0807 18:38:44.732026 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:44.732029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:44.732099 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:44.732203 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:44.733474 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.734177 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.736363 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.736757 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.736877 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:44.740532 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:44.740536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:44.740539 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:44.740572 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:44.742334 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.743086 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.743095 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.743096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.743190 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.743653 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.743659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.743662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.743933 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:44.744026 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:44.744606 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 18:38:44.744773 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:38:44.745169 6697 external_mini_cluster.cc:1428] Setting key 68e991ef9fc544fcf57fe9ccebe63739
I20260807 18:38:44.745875 18845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:44.729767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38003 (local address 127.6.138.65:57513)
0807 18:38:44.730293 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:44.730356 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:44.731544 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:44.732456 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:44.732477 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:44.732826 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:44.733318 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:44.733331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.734350 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.734354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:44.736193 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:44.736209 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.736927 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.736932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:44.737118 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:44.738035 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:44.738056 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:44.740311 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:44.742468 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:44.742478 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:44.742498 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:44.742930 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:44.743296 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:44.743299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:44.743302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:44.743509 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:44.744019 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:44.744024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:44.744293 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 18:38:44.744476 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 18:38:44.744833 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
I20260807 18:38:44.748919 19007 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38003
I20260807 18:38:44.749092 19007 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:44.749713 19007 heartbeater.cc:511] Master 127.6.138.126:38003 requested a full tablet report, sending...
I20260807 18:38:44.751518 18704 ts_manager.cc:194] Registered new tserver with Master: 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273)
I20260807 18:38:44.753073 18704 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:57513
WARNING: no policy specified for oryx/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:38:44.759454 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:44.751225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:54445 (local address 127.6.138.126:38003)
0807 18:38:44.751339 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:44.751344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:44.751621 (+ 277us) server_negotiation.cc:407] Connection header received
0807 18:38:44.751738 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:44.751741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:44.751789 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:44.751887 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:44.752562 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.753280 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.754674 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.754979 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.755104 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:44.756423 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:44.756426 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:44.756428 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:44.756451 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:44.757935 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.758438 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.758441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.758443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.758518 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.758772 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.758774 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.758775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.758940 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:44.759014 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:44.759243 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 18:38:44.759338 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:38:44.759687 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:44.751151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38003 (local address 127.6.138.65:54445)
0807 18:38:44.751520 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:44.751537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:44.751645 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:44.751932 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:44.751937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:44.752210 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:44.752416 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:44.752429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.753431 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.753438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:44.754535 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:44.754548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.755122 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.755129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:44.755236 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:44.755768 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:44.755785 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:44.756283 (+ 498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:44.758058 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:44.758061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:44.758062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:44.758358 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:44.758606 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:44.758608 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:44.758609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:44.758691 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:44.759021 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:44.759023 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:44.759101 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 18:38:44.759184 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 18:38:44.759409 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":92,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab.
Entry for principal oryx/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab.
I20260807 18:38:44.779652 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38003
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab
--principal=oryx/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:44.898054 19019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:44.898335 19019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:44.898380 19019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:44.902487 19019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:44.902549 19019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:44.902587 19019 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:38:44.902621 19019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:44.907815 19019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab
--principal=oryx/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.138.126:38003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:44.908905 19019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:44.909904 19019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:44.920723 19030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:44.920751 19026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:44.920758 19028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:44.921276 19019 server_base.cc:1034] running on GCE node
I20260807 18:38:44.921715 19019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:44.922389 19019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:44.923596 19019 hybrid_clock.cc:648] HybridClock initialized: now 1786127924923561 us; error 47 us; skew 500 ppm
Aug 07 18:38:44 dist-test-slave-2d19 krb5kdc[18651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127924, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:44.926580 19019 init.cc:377] Logged in from keytab as oryx/127.6.138.66@KRBTEST.COM (short username oryx)
I20260807 18:38:44.927574 19019 webserver.cc:460] Webserver started at http://127.6.138.66:38015/ using document root <none> and password file <none>
I20260807 18:38:44.928174 19019 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:44.928289 19019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:44.928540 19019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:44.930725 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "6d24058590b9402ba68586e937173af4"
format_stamp: "Formatted at 2026-08-07 18:38:44 on dist-test-slave-2d19"
server_key: "8485c6af16280a85b2b54f6e5b4893d1"
server_key_iv: "11016db364fe15b887b0a50b0a0c9f24"
server_key_version: "encryptionkey@0"
I20260807 18:38:44.931325 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "6d24058590b9402ba68586e937173af4"
format_stamp: "Formatted at 2026-08-07 18:38:44 on dist-test-slave-2d19"
server_key: "8485c6af16280a85b2b54f6e5b4893d1"
server_key_iv: "11016db364fe15b887b0a50b0a0c9f24"
server_key_version: "encryptionkey@0"
I20260807 18:38:44.935240 19019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 18:38:44.937856 19039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:44.939152 19019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 18:38:44.939321 19019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "6d24058590b9402ba68586e937173af4"
format_stamp: "Formatted at 2026-08-07 18:38:44 on dist-test-slave-2d19"
server_key: "8485c6af16280a85b2b54f6e5b4893d1"
server_key_iv: "11016db364fe15b887b0a50b0a0c9f24"
server_key_version: "encryptionkey@0"
I20260807 18:38:44.939448 19019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:44.958518 19019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:44.962252 19019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:44.962473 19019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:44.964121 19019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:44.964200 19019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:44.964345 19019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:44.964401 19019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:38:44 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127924, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:44.976967 19019 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:44237
I20260807 18:38:44.977679 19204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:44237 every 8 connection(s)
I20260807 18:38:44.978132 19019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:44.979774 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:44.964923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:53281 (local address 127.6.138.126:38003)
0807 18:38:44.965075 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:44.965079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:44.966076 (+ 997us) server_negotiation.cc:407] Connection header received
0807 18:38:44.967163 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:44.967167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:44.967237 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:44.967355 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:44.968880 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.969844 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.971325 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.971558 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.971668 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:44.975672 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:44.975676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:44.975679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:44.975714 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:44.977495 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.978109 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.978112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.978113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.978216 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.978584 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.978589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.978590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.978834 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:44.978944 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:44.979513 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 18:38:44.979644 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:38:44.980686 19055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:44.965374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38003 (local address 127.6.138.66:53281)
0807 18:38:44.965843 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:44.965894 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:44.966916 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:44.967659 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:44.967668 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:44.968121 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:44.968729 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:44.968744 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.970043 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.970046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:44.971193 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:44.971203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.971712 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.971716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:44.971935 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:44.972604 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:44.972623 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:44.975433 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:44.977613 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:44.977620 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:44.977636 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:44.977997 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:44.978311 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:44.978315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:44.978317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:44.978483 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:44.979003 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:44.979010 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:44.979254 (+ 244us) client_negotiation.cc:769] Sending connection context
0807 18:38:44.979416 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:38:44.979782 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":140,"threads_started":1}
I20260807 18:38:44.982071 19205 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38003
I20260807 18:38:44.982241 19205 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:44.982802 19205 heartbeater.cc:511] Master 127.6.138.126:38003 requested a full tablet report, sending...
I20260807 18:38:44.983959 18704 ts_manager.cc:194] Registered new tserver with Master: 6d24058590b9402ba68586e937173af4 (127.6.138.66:44237)
I20260807 18:38:44.984828 18704 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:53281
I20260807 18:38:44.987443 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 19019
I20260807 18:38:44.987581 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:44.987884 6697 external_mini_cluster.cc:1428] Setting key aeafec853c0220af989f65447162b9fb
I20260807 18:38:44.991950 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:44.984229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:43573 (local address 127.6.138.126:38003)
0807 18:38:44.984374 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:44.984378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:44.984606 (+ 228us) server_negotiation.cc:407] Connection header received
0807 18:38:44.984697 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:44.984699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:44.984737 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:44.984809 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:44.985301 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.985839 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.986722 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.986933 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.987046 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:44.988238 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:44.988241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:44.988243 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:44.988303 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:44.990106 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.990625 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.990634 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.990636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.990713 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:44.991007 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:44.991012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:44.991014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:44.991230 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:44.991352 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:44.991644 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 18:38:44.991764 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:38:44.991966 19213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:44.984188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38003 (local address 127.6.138.66:43573)
0807 18:38:44.984494 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:44.984510 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:44.984616 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:44.984854 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:44.984857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:44.985095 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:44.985211 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:44.985217 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.985969 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.985972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:44.986598 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:44.986608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:44.987045 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:44.987048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:44.987149 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:44.987719 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:44.987738 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:44.988083 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:44.990243 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:44.990247 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:44.990250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:44.990498 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:44.990809 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:44.990813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:44.990815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:44.990910 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:44.991339 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:44.991342 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:44.991458 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 18:38:44.991570 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 18:38:44.991731 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for oryx/127.6.138.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab.
Entry for principal oryx/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab.
I20260807 18:38:45.019342 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38003
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab
--principal=oryx/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:45.137645 19217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:45.137938 19217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:45.137986 19217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:45.142064 19217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:45.142134 19217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:45.142198 19217 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:38:45.142267 19217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:45.147826 19217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab
--principal=oryx/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.138.126:38003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:45.149039 19217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:45.150134 19217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:45.160020 19226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:45.159998 19230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:45.160056 19225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:45.160308 19217 server_base.cc:1034] running on GCE node
I20260807 18:38:45.160804 19217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:45.161456 19217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:45.162652 19217 hybrid_clock.cc:648] HybridClock initialized: now 1786127925162638 us; error 25 us; skew 500 ppm
Aug 07 18:38:45 dist-test-slave-2d19 krb5kdc[18651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127925, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:45.165628 19217 init.cc:377] Logged in from keytab as oryx/127.6.138.67@KRBTEST.COM (short username oryx)
I20260807 18:38:45.166646 19217 webserver.cc:460] Webserver started at http://127.6.138.67:35653/ using document root <none> and password file <none>
I20260807 18:38:45.167256 19217 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:45.167347 19217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:45.167588 19217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:45.169847 19217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "6f8afcf0879347a0b32934c8268b1fd0"
format_stamp: "Formatted at 2026-08-07 18:38:45 on dist-test-slave-2d19"
server_key: "5eb3ccee903e3eba97aa8029c171802a"
server_key_iv: "be0abef93d8579f13f380d90e782d08c"
server_key_version: "encryptionkey@0"
I20260807 18:38:45.170439 19217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "6f8afcf0879347a0b32934c8268b1fd0"
format_stamp: "Formatted at 2026-08-07 18:38:45 on dist-test-slave-2d19"
server_key: "5eb3ccee903e3eba97aa8029c171802a"
server_key_iv: "be0abef93d8579f13f380d90e782d08c"
server_key_version: "encryptionkey@0"
I20260807 18:38:45.174582 19217 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:45.177351 19240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:45.178669 19217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:45.178819 19217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "6f8afcf0879347a0b32934c8268b1fd0"
format_stamp: "Formatted at 2026-08-07 18:38:45 on dist-test-slave-2d19"
server_key: "5eb3ccee903e3eba97aa8029c171802a"
server_key_iv: "be0abef93d8579f13f380d90e782d08c"
server_key_version: "encryptionkey@0"
I20260807 18:38:45.179046 19217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:45.195530 19217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:45.198949 19217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:45.199158 19217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:45.200769 19217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:45.200820 19217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:45.200879 19217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:45.200915 19217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:38:45 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127925, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:45.212462 19217 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:43059
I20260807 18:38:45.212957 19413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:43059 every 8 connection(s)
I20260807 18:38:45.213539 19217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:45.215981 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.201529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:52547 (local address 127.6.138.126:38003)
0807 18:38:45.201657 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.201660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.202787 (+ 1127us) server_negotiation.cc:407] Connection header received
0807 18:38:45.203935 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.203938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.204001 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.204093 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:45.205517 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.206059 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.207364 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.207664 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.207781 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.211644 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:45.211648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:45.211650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:45.211679 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:45.213788 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:45.214470 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:45.214475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:45.214477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:45.214578 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:45.214946 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:45.214949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:45.214950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:45.215155 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:45.215241 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.215741 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.215861 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:38:45.216938 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.202129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38003 (local address 127.6.138.67:52547)
0807 18:38:45.202629 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.202674 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.203706 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.204492 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.204500 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.204829 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:45.205354 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.205378 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.206195 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.206199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.207235 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.207246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.207852 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.207856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.208029 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.208719 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:45.208742 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:45.211421 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:45.213970 (+ 2549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:45.213976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:45.213993 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:45.214343 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:45.214697 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:45.214700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:45.214702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:45.214843 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:45.215248 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:45.215253 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:45.215486 (+ 233us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.215685 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.216055 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":138,"threads_started":1}
I20260807 18:38:45.217213 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 19217
I20260807 18:38:45.217360 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:45.217717 6697 external_mini_cluster.cc:1428] Setting key 7499e6c4ba141490bd80aa03eb5baa00
I20260807 18:38:45.218240 19416 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38003
I20260807 18:38:45.218402 19416 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:45.218973 19416 heartbeater.cc:511] Master 127.6.138.126:38003 requested a full tablet report, sending...
I20260807 18:38:45.220077 18709 ts_manager.cc:194] Registered new tserver with Master: 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059)
I20260807 18:38:45.220775 18709 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:52547
I20260807 18:38:45.228245 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.220684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38003 (local address 127.6.138.67:50891)
0807 18:38:45.220948 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.220962 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.221042 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.221369 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.221372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.221596 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:45.221695 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.221701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.222667 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.222670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.223240 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.223248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.223734 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.223737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.223822 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.224372 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:45.224386 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:45.224813 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:45.226669 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:45.226673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:45.226674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:45.226879 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:45.227232 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:45.227235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:45.227236 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:45.227322 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:45.227737 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:45.227739 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:45.227820 (+ 81us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.227975 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.228102 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
I20260807 18:38:45.228344 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.221050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:50891 (local address 127.6.138.126:38003)
0807 18:38:45.221167 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.221172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.221185 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:38:45.221220 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.221223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.221267 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.221329 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:45.221812 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.222538 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.223350 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.223589 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.223674 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.225016 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:45.225019 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:45.225021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:45.225043 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:45.226557 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:45.227023 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:45.227026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:45.227028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:45.227126 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:45.227419 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:45.227422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:45.227423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:45.227616 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:45.227710 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.228074 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.228174 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 18:38:45.232071 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:38:45 dist-test-slave-2d19 krb5kdc[18651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:38:45 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:45 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:38:45.259656 18783 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:45.253508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41538 (local address 127.6.138.126:38003)
0807 18:38:45.253638 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.253641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.253655 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:38:45.253701 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.253703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.253745 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.253819 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:45.254623 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.255161 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.255986 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.256207 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.256372 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.259516 (+ 3144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41538: BlockingRecv error: recv got EOF from 127.0.0.1:41538 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 18:38:45 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127925, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:45.269649 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.261279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41540 (local address 127.6.138.126:38003)
0807 18:38:45.261417 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.261422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.261555 (+ 133us) server_negotiation.cc:407] Connection header received
0807 18:38:45.261650 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.261652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.261698 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.261774 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:45.262322 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.262820 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.263710 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.263932 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.264013 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.266278 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:45.266281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:45.266283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:45.266307 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:45.267745 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:45.268125 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:45.268128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:45.268129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:45.268212 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:45.268578 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:45.268581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:45.268582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:45.268767 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:45.268956 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.269315 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.269456 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:38:45.273233 18709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:45.276767 18709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:45.292223 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.286790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37273 (local address 127.0.0.1:60112)
0807 18:38:45.287668 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.287683 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.287777 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.288486 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.288489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.288508 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:45.288591 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.288598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.289728 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.289732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.290705 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.290715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.291796 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.291799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.291888 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.291922 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.292006 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.292103 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":1,"thread_start_us":102,"threads_started":1}
I20260807 18:38:45.292330 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.287340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.6.138.65:37273)
0807 18:38:45.288017 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.288023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.288044 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:38:45.288126 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.288131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.288351 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.288489 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:45.288711 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.289589 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.290860 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.291678 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.291794 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.291957 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.292063 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.292171 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":73,"threads_started":1}
I20260807 18:38:45.293056 19461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.285844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:44237 (local address 127.0.0.1:59828)
0807 18:38:45.287035 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.287052 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.287177 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.287719 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.287722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.287742 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:45.287852 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.287859 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.289061 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.289064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.290677 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.290692 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.292630 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.292634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.292727 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.292750 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.292841 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.292916 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"mutex_wait_us":55,"thread_start_us":145,"threads_started":1}
I20260807 18:38:45.293129 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.286568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59828 (local address 127.6.138.66:44237)
0807 18:38:45.287013 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.287025 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.287277 (+ 252us) server_negotiation.cc:407] Connection header received
0807 18:38:45.287366 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.287371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.287578 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.287744 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:45.287973 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.288952 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.290816 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.291679 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.291794 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.291975 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.292892 (+ 917us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.292987 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":130,"threads_started":1}
I20260807 18:38:45.293936 19454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.285760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43059 (local address 127.0.0.1:45000)
0807 18:38:45.286265 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.286320 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.286473 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.287520 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.287526 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.287552 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:45.287718 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.287729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.289190 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.289193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.290700 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.290707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.293657 (+ 2950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.293660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.293727 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.293745 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.293798 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.293845 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":151,"thread_start_us":83,"threads_started":1}
I20260807 18:38:45.294081 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.285867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.6.138.67:43059)
0807 18:38:45.286804 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.286818 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.286843 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:38:45.286933 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.286938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.287163 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.287326 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:45.287862 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.289065 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.292041 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.293536 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.293659 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.293816 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.293864 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.293949 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":124,"threads_started":1}
I20260807 18:38:45.296799 19310 tablet_service.cc:1467] Processing CreateTablet for tablet ca15a63fa18a41da9adbdc8512e7cd1f (DEFAULT_TABLE table=test-table [id=8b578a464ad64affbcff12b509c06702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:45.296788 18914 tablet_service.cc:1467] Processing CreateTablet for tablet ca15a63fa18a41da9adbdc8512e7cd1f (DEFAULT_TABLE table=test-table [id=8b578a464ad64affbcff12b509c06702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:45.297869 19310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca15a63fa18a41da9adbdc8512e7cd1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:45.297888 18914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca15a63fa18a41da9adbdc8512e7cd1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:45.298416 19110 tablet_service.cc:1467] Processing CreateTablet for tablet ca15a63fa18a41da9adbdc8512e7cd1f (DEFAULT_TABLE table=test-table [id=8b578a464ad64affbcff12b509c06702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:45.299479 19110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca15a63fa18a41da9adbdc8512e7cd1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:45.304389 19463 tablet_bootstrap.cc:492] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: Bootstrap starting.
I20260807 18:38:45.305737 19464 tablet_bootstrap.cc:492] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: Bootstrap starting.
I20260807 18:38:45.306721 19463 tablet_bootstrap.cc:654] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:45.307499 19463 log.cc:824] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:45.307789 19465 tablet_bootstrap.cc:492] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: Bootstrap starting.
I20260807 18:38:45.307901 19464 tablet_bootstrap.cc:654] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:45.308794 19464 log.cc:824] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:45.309965 19465 tablet_bootstrap.cc:654] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:45.310124 19463 tablet_bootstrap.cc:492] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: No bootstrap required, opened a new log
I20260807 18:38:45.310328 19463 ts_tablet_manager.cc:1397] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:38:45.310719 19465 log.cc:824] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:45.310806 19464 tablet_bootstrap.cc:492] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: No bootstrap required, opened a new log
I20260807 18:38:45.310982 19464 ts_tablet_manager.cc:1397] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:38:45.312723 19465 tablet_bootstrap.cc:492] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: No bootstrap required, opened a new log
I20260807 18:38:45.312958 19465 ts_tablet_manager.cc:1397] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:38:45.314550 19463 raft_consensus.cc:348] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } }
I20260807 18:38:45.314930 19463 raft_consensus.cc:374] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:45.315047 19463 raft_consensus.cc:729] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8afcf0879347a0b32934c8268b1fd0, State: Initialized, Role: FOLLOWER
I20260807 18:38:45.315627 19463 consensus_queue.cc:260] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } }
I20260807 18:38:45.315559 19464 raft_consensus.cc:348] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } }
I20260807 18:38:45.315801 19464 raft_consensus.cc:374] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:45.315876 19464 raft_consensus.cc:729] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173662d55dd946608a6e0ffd293a2bb4, State: Initialized, Role: FOLLOWER
I20260807 18:38:45.316449 19416 heartbeater.cc:503] Master 127.6.138.126:38003 was elected leader, sending a full tablet report...
I20260807 18:38:45.316429 19464 consensus_queue.cc:260] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } }
I20260807 18:38:45.317118 19007 heartbeater.cc:503] Master 127.6.138.126:38003 was elected leader, sending a full tablet report...
I20260807 18:38:45.317384 19464 ts_tablet_manager.cc:1428] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:38:45.317902 19463 ts_tablet_manager.cc:1428] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: Time spent starting tablet: real 0.008s user 0.000s sys 0.005s
I20260807 18:38:45.318140 19465 raft_consensus.cc:348] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } }
I20260807 18:38:45.318393 19465 raft_consensus.cc:374] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:45.318492 19465 raft_consensus.cc:729] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d24058590b9402ba68586e937173af4, State: Initialized, Role: FOLLOWER
I20260807 18:38:45.318943 19465 consensus_queue.cc:260] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } }
I20260807 18:38:45.319875 19205 heartbeater.cc:503] Master 127.6.138.126:38003 was elected leader, sending a full tablet report...
I20260807 18:38:45.319999 19465 ts_tablet_manager.cc:1428] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 18:38:45.439558 19471 raft_consensus.cc:482] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:45.439870 19471 raft_consensus.cc:504] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } }
I20260807 18:38:45.441160 19471 leader_election.cc:288] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059), 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273)
I20260807 18:38:45.445873 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.441654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:39203 (local address 127.6.138.67:43059)
0807 18:38:45.441995 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.442003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.442053 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:38:45.442168 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.442172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.442229 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.442321 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:45.442736 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.443420 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.444907 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.445309 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.445440 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.445499 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.445675 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.445746 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
I20260807 18:38:45.445900 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.441555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43059 (local address 127.6.138.66:39203)
0807 18:38:45.441962 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.441976 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.442072 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.442343 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.442346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.442386 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:45.442589 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.442598 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.443566 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.443570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.444781 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.444792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.445434 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.445437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.445541 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.445564 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.445657 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.445732 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":94,"spinlock_wait_cycles":8064,"thread_start_us":134,"threads_started":1}
I20260807 18:38:45.446375 19055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.441555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37273 (local address 127.6.138.66:58335)
0807 18:38:45.441768 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.441799 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.441918 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.442343 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.442346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.442385 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:45.442558 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.442570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.443703 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.443708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.445191 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.445203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.445948 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.445952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.446069 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.446091 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.446176 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.446262 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":4,"spinlock_wait_cycles":30336}
I20260807 18:38:45.446703 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.441878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:58335 (local address 127.6.138.65:37273)
0807 18:38:45.442103 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.442109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.442128 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:38:45.442180 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.442183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.442229 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.442321 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:45.442708 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.443539 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.445334 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.445810 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.445967 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.446018 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.446495 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.446585 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260807 18:38:45.446913 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca15a63fa18a41da9adbdc8512e7cd1f" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8afcf0879347a0b32934c8268b1fd0" is_pre_election: true
I20260807 18:38:45.447234 19344 raft_consensus.cc:2454] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 0.
I20260807 18:38:45.447665 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca15a63fa18a41da9adbdc8512e7cd1f" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173662d55dd946608a6e0ffd293a2bb4" is_pre_election: true
I20260807 18:38:45.447837 19047 leader_election.cc:302] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d24058590b9402ba68586e937173af4, 6f8afcf0879347a0b32934c8268b1fd0; no voters:
I20260807 18:38:45.448004 18939 raft_consensus.cc:2454] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 0.
I20260807 18:38:45.448163 19471 raft_consensus.cc:2781] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:45.448302 19471 raft_consensus.cc:482] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:45.448371 19471 raft_consensus.cc:3037] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:45.449595 19471 raft_consensus.cc:504] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } }
I20260807 18:38:45.449999 19471 leader_election.cc:288] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 election: Requested vote from peers 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059), 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273)
I20260807 18:38:45.450318 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca15a63fa18a41da9adbdc8512e7cd1f" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8afcf0879347a0b32934c8268b1fd0"
I20260807 18:38:45.450461 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca15a63fa18a41da9adbdc8512e7cd1f" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173662d55dd946608a6e0ffd293a2bb4"
I20260807 18:38:45.450492 19344 raft_consensus.cc:3037] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:45.450591 18939 raft_consensus.cc:3037] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:45.451907 18939 raft_consensus.cc:2454] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 1.
I20260807 18:38:45.451907 19344 raft_consensus.cc:2454] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 1.
I20260807 18:38:45.452280 19050 leader_election.cc:302] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173662d55dd946608a6e0ffd293a2bb4, 6d24058590b9402ba68586e937173af4; no voters:
I20260807 18:38:45.452498 19471 raft_consensus.cc:2781] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:45.452740 19471 raft_consensus.cc:686] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 1 LEADER]: Becoming Leader. State: Replica: 6d24058590b9402ba68586e937173af4, State: Running, Role: LEADER
I20260807 18:38:45.453136 19471 consensus_queue.cc:237] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } }
I20260807 18:38:45.456501 18704 catalog_manager.cc:5495] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d24058590b9402ba68586e937173af4 (127.6.138.66). New cstate: current_term: 1 leader_uuid: "6d24058590b9402ba68586e937173af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } health_report { overall_health: UNKNOWN } } }
W20260807 18:38:45.464989 19417 tablet.cc:2369] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:38:45.479521 19206 tablet.cc:2369] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:45.489960 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.485907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59838 (local address 127.6.138.66:44237)
0807 18:38:45.486026 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.486030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.486067 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:38:45.486168 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.486170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.486227 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.486319 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:45.486603 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.487470 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.488476 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.488708 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.488815 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.489097 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:45.489542 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:45.489634 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.489714 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.489768 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260807 18:38:45.494385 19008 tablet.cc:2369] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:38:45.496316 18939 raft_consensus.cc:1261] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Refusing update from remote peer 6d24058590b9402ba68586e937173af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:45.496380 19344 raft_consensus.cc:1261] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Refusing update from remote peer 6d24058590b9402ba68586e937173af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:45.497040 19474 consensus_queue.cc:1035] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:45.497227 19471 consensus_queue.cc:1035] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:45.504925 19480 mvcc.cc:205] Tried to move back new op lower bound from 7315979982824833024 to 7315979982659248128. Current Snapshot: MvccSnapshot[applied={T|T < 7315979982824833024}]
I20260807 18:38:45.506316 19485 mvcc.cc:205] Tried to move back new op lower bound from 7315979982824833024 to 7315979982659248128. Current Snapshot: MvccSnapshot[applied={T|T < 7315979982824833024}]
I20260807 18:38:45.514509 18709 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:45.514681 18709 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:45.517254 18709 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80: Sending DeleteTablet for 3 replicas of tablet ca15a63fa18a41da9adbdc8512e7cd1f
I20260807 18:38:45.518144 19310 tablet_service.cc:1514] Processing DeleteTablet for tablet ca15a63fa18a41da9adbdc8512e7cd1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:45 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45000
I20260807 18:38:45.518194 18914 tablet_service.cc:1514] Processing DeleteTablet for tablet ca15a63fa18a41da9adbdc8512e7cd1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:45 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:60112
I20260807 18:38:45.518878 19110 tablet_service.cc:1514] Processing DeleteTablet for tablet ca15a63fa18a41da9adbdc8512e7cd1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:45 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:59828
I20260807 18:38:45.518838 18709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:38:45.519073 19496 tablet_replica.cc:330] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: stopping tablet replica
W20260807 18:38:45.519292 18709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:45.519408 19496 raft_consensus.cc:2229] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:45.519657 19499 tablet_replica.cc:330] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: stopping tablet replica
I20260807 18:38:45.519783 19496 raft_consensus.cc:2258] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:45.520184 19498 tablet_replica.cc:330] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: stopping tablet replica
I20260807 18:38:45.520289 19499 raft_consensus.cc:2229] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:45.520745 19499 raft_consensus.cc:2258] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:45.521938 19496 ts_tablet_manager.cc:1905] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:45.522166 19499 ts_tablet_manager.cc:1905] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:45.524983 19496 ts_tablet_manager.cc:1918] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:45.525022 19110 tablet_service.cc:1467] Processing CreateTablet for tablet 882c3ac3ee23472280531a7cc6e95121 (DEFAULT_TABLE table=test-table [id=4b965028027b432093b789f31ed186a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:45.525174 19496 log.cc:1196] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/ca15a63fa18a41da9adbdc8512e7cd1f
I20260807 18:38:45.525228 19310 tablet_service.cc:1467] Processing CreateTablet for tablet 882c3ac3ee23472280531a7cc6e95121 (DEFAULT_TABLE table=test-table [id=4b965028027b432093b789f31ed186a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:45.525282 19110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882c3ac3ee23472280531a7cc6e95121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:45.525498 19310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882c3ac3ee23472280531a7cc6e95121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:45.525537 19496 ts_tablet_manager.cc:1939] T ca15a63fa18a41da9adbdc8512e7cd1f P 6f8afcf0879347a0b32934c8268b1fd0: Deleting consensus metadata
I20260807 18:38:45.526820 18692 catalog_manager.cc:4841] TS 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059): tablet ca15a63fa18a41da9adbdc8512e7cd1f (table test-table [id=8b578a464ad64affbcff12b509c06702]) successfully deleted
I20260807 18:38:45.527585 19463 tablet_bootstrap.cc:492] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: Bootstrap starting.
I20260807 18:38:45.527840 19465 tablet_bootstrap.cc:492] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: Bootstrap starting.
I20260807 18:38:45.528669 19463 tablet_bootstrap.cc:654] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:45.528880 19465 tablet_bootstrap.cc:654] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:45.524724 18914 tablet_service.cc:1467] Processing CreateTablet for tablet 882c3ac3ee23472280531a7cc6e95121 (DEFAULT_TABLE table=test-table [id=4b965028027b432093b789f31ed186a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:45.529201 19499 ts_tablet_manager.cc:1918] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:45.529296 19499 log.cc:1196] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/ca15a63fa18a41da9adbdc8512e7cd1f
I20260807 18:38:45.529433 18914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882c3ac3ee23472280531a7cc6e95121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:45.529762 19499 ts_tablet_manager.cc:1939] T ca15a63fa18a41da9adbdc8512e7cd1f P 6d24058590b9402ba68586e937173af4: Deleting consensus metadata
I20260807 18:38:45.530206 19465 tablet_bootstrap.cc:492] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: No bootstrap required, opened a new log
I20260807 18:38:45.530300 19465 ts_tablet_manager.cc:1397] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 18:38:45.530774 19465 raft_consensus.cc:348] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.530905 19465 raft_consensus.cc:374] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:45.530957 19465 raft_consensus.cc:729] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d24058590b9402ba68586e937173af4, State: Initialized, Role: FOLLOWER
I20260807 18:38:45.531109 19465 consensus_queue.cc:260] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.531267 18691 catalog_manager.cc:4841] TS 6d24058590b9402ba68586e937173af4 (127.6.138.66:44237): tablet ca15a63fa18a41da9adbdc8512e7cd1f (table test-table [id=8b578a464ad64affbcff12b509c06702]) successfully deleted
I20260807 18:38:45.531437 19465 ts_tablet_manager.cc:1428] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 18:38:45.532380 19464 tablet_bootstrap.cc:492] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: Bootstrap starting.
I20260807 18:38:45.532900 19498 raft_consensus.cc:2229] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:45.533252 19498 raft_consensus.cc:2258] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:45.533389 19464 tablet_bootstrap.cc:654] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:45.534651 19464 tablet_bootstrap.cc:492] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: No bootstrap required, opened a new log
I20260807 18:38:45.534754 19464 ts_tablet_manager.cc:1397] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 18:38:45.534919 19498 ts_tablet_manager.cc:1905] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:45.535194 19464 raft_consensus.cc:348] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.535298 19464 raft_consensus.cc:374] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:45.535329 19464 raft_consensus.cc:729] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173662d55dd946608a6e0ffd293a2bb4, State: Initialized, Role: FOLLOWER
I20260807 18:38:45.535463 19464 consensus_queue.cc:260] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.536857 19464 ts_tablet_manager.cc:1428] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 18:38:45.538367 19498 ts_tablet_manager.cc:1918] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:45.538492 19498 log.cc:1196] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/ca15a63fa18a41da9adbdc8512e7cd1f
I20260807 18:38:45.538861 19498 ts_tablet_manager.cc:1939] T ca15a63fa18a41da9adbdc8512e7cd1f P 173662d55dd946608a6e0ffd293a2bb4: Deleting consensus metadata
I20260807 18:38:45.540158 18695 catalog_manager.cc:4841] TS 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273): tablet ca15a63fa18a41da9adbdc8512e7cd1f (table test-table [id=8b578a464ad64affbcff12b509c06702]) successfully deleted
I20260807 18:38:45.540879 19463 tablet_bootstrap.cc:492] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: No bootstrap required, opened a new log
I20260807 18:38:45.541000 19463 ts_tablet_manager.cc:1397] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260807 18:38:45.541447 19463 raft_consensus.cc:348] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.541566 19463 raft_consensus.cc:374] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:45.541589 19463 raft_consensus.cc:729] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8afcf0879347a0b32934c8268b1fd0, State: Initialized, Role: FOLLOWER
I20260807 18:38:45.541749 19463 consensus_queue.cc:260] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.542050 19463 ts_tablet_manager.cc:1428] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:38:45.589354 19469 raft_consensus.cc:482] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:45.589511 19469 raft_consensus.cc:504] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.590737 19469 leader_election.cc:288] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273), 6d24058590b9402ba68586e937173af4 (127.6.138.66:44237)
I20260807 18:38:45.595566 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.591015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:44237 (local address 127.6.138.67:36773)
0807 18:38:45.591341 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.591353 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.591442 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.591773 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.591776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.591801 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:45.591971 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.592020 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.593075 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.593079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.594215 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.594226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.595089 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.595093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.595200 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.595230 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.595312 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.595390 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":88,"threads_started":1}
I20260807 18:38:45.595726 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.590935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37273 (local address 127.6.138.67:60053)
0807 18:38:45.591069 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.591091 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.591199 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.591977 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.591980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.591995 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:45.592093 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.592101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.592853 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.592858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.594176 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.594189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.595269 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.595275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.595373 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.595391 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.595457 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.595533 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":40}
I20260807 18:38:45.595779 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.591407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:60053 (local address 127.6.138.65:37273)
0807 18:38:45.591654 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.591658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.591671 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:38:45.591738 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.591741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.591789 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.591874 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:45.592195 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.592740 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.594317 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.595021 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.595138 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.595196 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.595554 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.595645 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260807 18:38:45.596230 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882c3ac3ee23472280531a7cc6e95121" candidate_uuid: "6f8afcf0879347a0b32934c8268b1fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173662d55dd946608a6e0ffd293a2bb4" is_pre_election: true
I20260807 18:38:45.596433 18939 raft_consensus.cc:2454] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8afcf0879347a0b32934c8268b1fd0 in term 0.
I20260807 18:38:45.596472 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.591385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:36773 (local address 127.6.138.66:44237)
0807 18:38:45.591501 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.591506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.591521 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:38:45.591591 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.591594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.591656 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.591742 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:45.592143 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.592960 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.594392 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.594945 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.595089 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.595149 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.596247 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.596354 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:38:45.597016 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882c3ac3ee23472280531a7cc6e95121" candidate_uuid: "6f8afcf0879347a0b32934c8268b1fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d24058590b9402ba68586e937173af4" is_pre_election: true
I20260807 18:38:45.597047 19244 leader_election.cc:302] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173662d55dd946608a6e0ffd293a2bb4, 6f8afcf0879347a0b32934c8268b1fd0; no voters:
I20260807 18:38:45.597278 19139 raft_consensus.cc:2454] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f8afcf0879347a0b32934c8268b1fd0 in term 0.
I20260807 18:38:45.597327 19469 raft_consensus.cc:2781] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:45.597378 19469 raft_consensus.cc:482] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:45.597414 19469 raft_consensus.cc:3037] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:45.598743 19469 raft_consensus.cc:504] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.599144 19469 leader_election.cc:288] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [CANDIDATE]: Term 1 election: Requested vote from peers 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273), 6d24058590b9402ba68586e937173af4 (127.6.138.66:44237)
I20260807 18:38:45.599570 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882c3ac3ee23472280531a7cc6e95121" candidate_uuid: "6f8afcf0879347a0b32934c8268b1fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d24058590b9402ba68586e937173af4"
I20260807 18:38:45.599649 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882c3ac3ee23472280531a7cc6e95121" candidate_uuid: "6f8afcf0879347a0b32934c8268b1fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173662d55dd946608a6e0ffd293a2bb4"
I20260807 18:38:45.599704 19139 raft_consensus.cc:3037] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:45.599785 18939 raft_consensus.cc:3037] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:45.600849 19139 raft_consensus.cc:2454] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8afcf0879347a0b32934c8268b1fd0 in term 1.
I20260807 18:38:45.600981 18939 raft_consensus.cc:2454] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f8afcf0879347a0b32934c8268b1fd0 in term 1.
I20260807 18:38:45.601322 19244 leader_election.cc:302] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173662d55dd946608a6e0ffd293a2bb4, 6f8afcf0879347a0b32934c8268b1fd0; no voters:
I20260807 18:38:45.601557 19469 raft_consensus.cc:2781] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:45.601786 19469 raft_consensus.cc:686] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 LEADER]: Becoming Leader. State: Replica: 6f8afcf0879347a0b32934c8268b1fd0, State: Running, Role: LEADER
I20260807 18:38:45.602051 19469 consensus_queue.cc:237] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.604636 18704 catalog_manager.cc:5495] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "6f8afcf0879347a0b32934c8268b1fd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } health_report { overall_health: HEALTHY } } }
Aug 07 18:38:45 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127924, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.126@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:45.652177 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.642231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41550 (local address 127.6.138.126:38003)
0807 18:38:45.642381 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.642386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.642625 (+ 239us) server_negotiation.cc:407] Connection header received
0807 18:38:45.642815 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.642817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.642855 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.642934 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:45.643670 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.644564 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.645382 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.645725 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.645852 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.648486 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:45.648489 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:45.648491 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:45.648514 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:45.650019 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:45.650603 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:45.650605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:45.650607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:45.650680 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:45.651090 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:45.651094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:45.651095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:45.651325 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:45.651443 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.651832 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.652009 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":46}
I20260807 18:38:45.652190 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.642180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38003 (local address 127.0.0.1:41550)
0807 18:38:45.642537 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.642549 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.642687 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.642952 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.642955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.643404 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:45.643506 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.643513 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.644684 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.644686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.645246 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.645254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.645851 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.645853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.645953 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.646593 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:45.646605 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:45.648329 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:45.650130 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:45.650141 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:45.650148 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:45.650486 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:45.650775 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:45.650778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:45.650780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:45.650890 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:45.651463 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:45.651472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:45.651626 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.651762 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.651944 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":22,"thread_start_us":133,"threads_started":1}
I20260807 18:38:45.657130 18704 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:41550:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:38:45.657943 18704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:45.663004 18914 tablet_service.cc:1467] Processing CreateTablet for tablet 3089b3187dc54800b46abc304d27f838 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3226f1a41f2a4255a445846e6687ebc1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:38:45.663239 19310 tablet_service.cc:1467] Processing CreateTablet for tablet 3089b3187dc54800b46abc304d27f838 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3226f1a41f2a4255a445846e6687ebc1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:38:45.663417 18914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3089b3187dc54800b46abc304d27f838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:45.663659 19310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3089b3187dc54800b46abc304d27f838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:45.666253 19463 tablet_bootstrap.cc:492] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: Bootstrap starting.
I20260807 18:38:45.666893 19110 tablet_service.cc:1467] Processing CreateTablet for tablet 3089b3187dc54800b46abc304d27f838 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3226f1a41f2a4255a445846e6687ebc1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:38:45.667241 19110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3089b3187dc54800b46abc304d27f838. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:45.667335 19463 tablet_bootstrap.cc:654] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:45.669013 19463 tablet_bootstrap.cc:492] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: No bootstrap required, opened a new log
I20260807 18:38:45.669121 19463 ts_tablet_manager.cc:1397] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:45.669615 19463 raft_consensus.cc:348] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.669781 19463 raft_consensus.cc:374] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:45.669823 19463 raft_consensus.cc:729] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8afcf0879347a0b32934c8268b1fd0, State: Initialized, Role: FOLLOWER
I20260807 18:38:45.670038 19465 tablet_bootstrap.cc:492] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: Bootstrap starting.
I20260807 18:38:45.670109 19463 consensus_queue.cc:260] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.670665 19463 ts_tablet_manager.cc:1428] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:45.670982 19464 tablet_bootstrap.cc:492] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: Bootstrap starting.
I20260807 18:38:45.671228 19465 tablet_bootstrap.cc:654] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:45.671969 19464 tablet_bootstrap.cc:654] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:45.670368 19469 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.672690 19469 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:45.673863 19464 tablet_bootstrap.cc:492] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: No bootstrap required, opened a new log
I20260807 18:38:45.673967 19464 ts_tablet_manager.cc:1397] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:45.674223 19465 tablet_bootstrap.cc:492] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: No bootstrap required, opened a new log
I20260807 18:38:45.674331 19465 ts_tablet_manager.cc:1397] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260807 18:38:45.674492 19464 raft_consensus.cc:348] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.674603 19464 raft_consensus.cc:374] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:45.674636 19464 raft_consensus.cc:729] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173662d55dd946608a6e0ffd293a2bb4, State: Initialized, Role: FOLLOWER
I20260807 18:38:45.674765 19464 consensus_queue.cc:260] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.674787 19465 raft_consensus.cc:348] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.674888 19465 raft_consensus.cc:374] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:45.674921 19465 raft_consensus.cc:729] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d24058590b9402ba68586e937173af4, State: Initialized, Role: FOLLOWER
I20260807 18:38:45.675105 19464 ts_tablet_manager.cc:1428] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:45.675042 19465 consensus_queue.cc:260] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.675055 19470 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.675232 19470 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:45.675987 19490 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.676139 19490 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:45.676466 19465 ts_tablet_manager.cc:1428] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:45.862057 19470 raft_consensus.cc:482] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:45.862214 19470 raft_consensus.cc:504] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.863355 19470 leader_election.cc:288] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d24058590b9402ba68586e937173af4 (127.6.138.66:44237), 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059)
I20260807 18:38:45.867359 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.863739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:33139 (local address 127.6.138.67:43059)
0807 18:38:45.863912 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.863917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.864178 (+ 261us) server_negotiation.cc:407] Connection header received
0807 18:38:45.864395 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.864398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.864449 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.864532 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:45.864794 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.865311 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.866394 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.866793 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.866918 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.866980 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.867162 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.867242 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:38:45.867381 19521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.863635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43059 (local address 127.6.138.65:33139)
0807 18:38:45.864081 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.864094 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.864238 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.864556 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.864558 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.864577 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:45.864687 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.864692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.865431 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.865435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.866268 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.866279 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.866917 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.866921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.867031 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.867058 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.867144 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.867217 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":151,"threads_started":1}
I20260807 18:38:45.868013 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.863640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:44237 (local address 127.6.138.65:44749)
0807 18:38:45.864311 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.864330 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.864482 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.864770 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.864773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.864792 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:38:45.864921 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.864927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.865774 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.865778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.866656 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.866666 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.867389 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.867393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.867665 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.867691 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.867783 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.867868 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":198,"thread_start_us":228,"threads_started":1}
I20260807 18:38:45.867964 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3089b3187dc54800b46abc304d27f838" candidate_uuid: "173662d55dd946608a6e0ffd293a2bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8afcf0879347a0b32934c8268b1fd0" is_pre_election: true
I20260807 18:38:45.868048 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.863787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:44749 (local address 127.6.138.66:44237)
0807 18:38:45.863918 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.863925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.864439 (+ 514us) server_negotiation.cc:407] Connection header received
0807 18:38:45.864616 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.864619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.864669 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.864757 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:38:45.865050 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.865631 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.866801 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.867238 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.867392 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.867445 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.867835 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.867917 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:38:45.868144 19344 raft_consensus.cc:2454] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 173662d55dd946608a6e0ffd293a2bb4 in term 0.
I20260807 18:38:45.868579 18838 leader_election.cc:302] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173662d55dd946608a6e0ffd293a2bb4, 6f8afcf0879347a0b32934c8268b1fd0; no voters:
I20260807 18:38:45.868678 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3089b3187dc54800b46abc304d27f838" candidate_uuid: "173662d55dd946608a6e0ffd293a2bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d24058590b9402ba68586e937173af4" is_pre_election: true
I20260807 18:38:45.868856 19139 raft_consensus.cc:2454] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 173662d55dd946608a6e0ffd293a2bb4 in term 0.
I20260807 18:38:45.868860 19470 raft_consensus.cc:2781] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:45.868928 19470 raft_consensus.cc:482] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:45.868957 19470 raft_consensus.cc:3037] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:45.870085 19470 raft_consensus.cc:504] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.870500 19470 leader_election.cc:288] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [CANDIDATE]: Term 1 election: Requested vote from peers 6d24058590b9402ba68586e937173af4 (127.6.138.66:44237), 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059)
I20260807 18:38:45.870944 19139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3089b3187dc54800b46abc304d27f838" candidate_uuid: "173662d55dd946608a6e0ffd293a2bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d24058590b9402ba68586e937173af4"
I20260807 18:38:45.871060 19139 raft_consensus.cc:3037] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:45.871989 19139 raft_consensus.cc:2454] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 173662d55dd946608a6e0ffd293a2bb4 in term 1.
I20260807 18:38:45.872289 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3089b3187dc54800b46abc304d27f838" candidate_uuid: "173662d55dd946608a6e0ffd293a2bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8afcf0879347a0b32934c8268b1fd0"
I20260807 18:38:45.872416 19344 raft_consensus.cc:3037] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:45.873142 18836 leader_election.cc:302] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 173662d55dd946608a6e0ffd293a2bb4, 6d24058590b9402ba68586e937173af4; no voters:
I20260807 18:38:45.873373 19344 raft_consensus.cc:2454] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 173662d55dd946608a6e0ffd293a2bb4 in term 1.
I20260807 18:38:45.873473 19470 raft_consensus.cc:2781] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:45.873802 19470 raft_consensus.cc:686] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [term 1 LEADER]: Becoming Leader. State: Replica: 173662d55dd946608a6e0ffd293a2bb4, State: Running, Role: LEADER
I20260807 18:38:45.874150 19470 consensus_queue.cc:237] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:45.876210 19523 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: TxnStatusTablet state changed. Reason: New leader 173662d55dd946608a6e0ffd293a2bb4. Latest consensus state: current_term: 1 leader_uuid: "173662d55dd946608a6e0ffd293a2bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.876387 19523 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:38:45.876705 19529 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:38:45.876874 19529 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:38:45.877365 18709 catalog_manager.cc:5495] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65). New cstate: current_term: 1 leader_uuid: "173662d55dd946608a6e0ffd293a2bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } health_report { overall_health: UNKNOWN } } }
I20260807 18:38:45.910462 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.906393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.6.138.65:37273)
0807 18:38:45.906536 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.906540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.906554 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:38:45.906751 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.906754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.906832 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.906925 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:45.907191 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.907685 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.908929 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.909153 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.909237 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.909573 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:45.910085 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:45.910179 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.910296 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.910349 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:38:45.910517 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.906179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37273 (local address 127.0.0.1:60122)
0807 18:38:45.906442 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.906470 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.906579 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.906959 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.906962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.906969 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:45.907084 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.907089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.907791 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.907795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.908806 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.908818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.909262 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.909266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.909359 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.909416 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:45.910181 (+ 765us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:45.910197 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.910271 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.910358 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260807 18:38:45.913623 19344 raft_consensus.cc:1261] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Refusing update from remote peer 173662d55dd946608a6e0ffd293a2bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:45.913796 19139 raft_consensus.cc:1261] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Refusing update from remote peer 173662d55dd946608a6e0ffd293a2bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:45.914170 19523 consensus_queue.cc:1035] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:45.914428 19470 consensus_queue.cc:1035] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:45.916060 19469 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: TxnStatusTablet state changed. Reason: New leader 173662d55dd946608a6e0ffd293a2bb4. Latest consensus state: current_term: 1 leader_uuid: "173662d55dd946608a6e0ffd293a2bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.916180 19469 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:45.916198 19490 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: TxnStatusTablet state changed. Reason: New leader 173662d55dd946608a6e0ffd293a2bb4. Latest consensus state: current_term: 1 leader_uuid: "173662d55dd946608a6e0ffd293a2bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.916333 19490 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:45.919308 19469 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "173662d55dd946608a6e0ffd293a2bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.919297 19470 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "173662d55dd946608a6e0ffd293a2bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.919422 19470 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:38:45.919423 19469 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 6f8afcf0879347a0b32934c8268b1fd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:45.922358 19470 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "173662d55dd946608a6e0ffd293a2bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.922520 19470 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 173662d55dd946608a6e0ffd293a2bb4: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:38:45.925153 19490 tablet_replica.cc:439] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "173662d55dd946608a6e0ffd293a2bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } } }
I20260807 18:38:45.925292 19490 tablet_replica.cc:442] T 3089b3187dc54800b46abc304d27f838 P 6d24058590b9402ba68586e937173af4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:38:45.931283 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.927254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.6.138.67:43059)
0807 18:38:45.927388 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.927393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.927407 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:38:45.927497 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.927500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.927565 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.927677 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:45.927903 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.928724 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.929549 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.929889 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.930015 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.930287 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:45.930867 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:45.930986 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.931064 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.931129 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:38:45.942530 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.938888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:51717 (local address 127.6.138.65:37273)
0807 18:38:45.939038 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.939044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.939226 (+ 182us) server_negotiation.cc:407] Connection header received
0807 18:38:45.939373 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.939375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.939422 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.939496 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:45.939746 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.940237 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.941532 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.941733 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.941812 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.942121 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:45.942192 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:45.942245 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.942359 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.942406 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:38:45.942596 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.938811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37273 (local address 127.6.138.67:51717)
0807 18:38:45.939134 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.939150 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.939259 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.939510 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.939514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.939521 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:45.939637 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.939643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.940401 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.940404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.941405 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.941417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.941839 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.941843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.941939 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.941993 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:45.942272 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:45.942279 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.942345 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.942425 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":114,"threads_started":1}
I20260807 18:38:45.950170 19139 raft_consensus.cc:1261] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Refusing update from remote peer 6f8afcf0879347a0b32934c8268b1fd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:45.950258 18939 raft_consensus.cc:1261] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Refusing update from remote peer 6f8afcf0879347a0b32934c8268b1fd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:45.950732 19505 consensus_queue.cc:1035] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:45.950947 19469 consensus_queue.cc:1035] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:45.958699 19482 mvcc.cc:205] Tried to move back new op lower bound from 7315979984687427584 to 7315979983268691968. Current Snapshot: MvccSnapshot[applied={T|T < 7315979984687427584}]
I20260807 18:38:45.976379 19549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.971852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43059 (local address 127.6.138.65:40327)
0807 18:38:45.972190 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:45.972203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:45.972343 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:45.972628 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:45.972633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:45.972644 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:45.972793 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.972801 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.973995 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.973998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.974884 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:45.974914 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.975542 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.975545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:45.975617 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.975683 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:45.976049 (+ 366us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:45.976056 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:38:45.976114 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 18:38:45.976182 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":118,"threads_started":1}
I20260807 18:38:45.976363 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:45.972012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:40327 (local address 127.6.138.67:43059)
0807 18:38:45.972122 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:45.972126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:45.972324 (+ 198us) server_negotiation.cc:407] Connection header received
0807 18:38:45.972440 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:45.972443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:45.972510 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:45.972607 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:45.972940 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.973876 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.975031 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:45.975413 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:45.975530 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:45.975826 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:45.975937 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:45.976017 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:45.976135 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 18:38:45.976194 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:38:46.010840 18709 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:46.011047 18709 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:46.013780 18709 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80: Sending DeleteTablet for 3 replicas of tablet 882c3ac3ee23472280531a7cc6e95121
I20260807 18:38:46.014443 18914 tablet_service.cc:1514] Processing DeleteTablet for tablet 882c3ac3ee23472280531a7cc6e95121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:46 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:60112
I20260807 18:38:46.014703 19498 tablet_replica.cc:330] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: stopping tablet replica
I20260807 18:38:46.014848 19498 raft_consensus.cc:2229] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:46.015004 19498 raft_consensus.cc:2258] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:46.015230 19110 tablet_service.cc:1514] Processing DeleteTablet for tablet 882c3ac3ee23472280531a7cc6e95121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:46 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:59828
I20260807 18:38:46.015452 19499 tablet_replica.cc:330] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: stopping tablet replica
I20260807 18:38:46.015602 19499 raft_consensus.cc:2229] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:46.015730 19499 raft_consensus.cc:2258] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:46.016310 19498 ts_tablet_manager.cc:1905] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:46.016693 19499 ts_tablet_manager.cc:1905] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:46.018603 19310 tablet_service.cc:1514] Processing DeleteTablet for tablet 882c3ac3ee23472280531a7cc6e95121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:46 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45000
I20260807 18:38:46.018785 19496 tablet_replica.cc:330] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: stopping tablet replica
I20260807 18:38:46.018893 19496 raft_consensus.cc:2229] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:46.019218 19496 raft_consensus.cc:2258] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:46.020058 19496 ts_tablet_manager.cc:1905] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:46.020522 19499 ts_tablet_manager.cc:1918] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:46.020608 19499 log.cc:1196] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/882c3ac3ee23472280531a7cc6e95121
I20260807 18:38:46.020933 19499 ts_tablet_manager.cc:1939] T 882c3ac3ee23472280531a7cc6e95121 P 6d24058590b9402ba68586e937173af4: Deleting consensus metadata
I20260807 18:38:46.022859 19496 ts_tablet_manager.cc:1918] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:46.022944 19496 log.cc:1196] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/882c3ac3ee23472280531a7cc6e95121
I20260807 18:38:46.023216 19496 ts_tablet_manager.cc:1939] T 882c3ac3ee23472280531a7cc6e95121 P 6f8afcf0879347a0b32934c8268b1fd0: Deleting consensus metadata
I20260807 18:38:46.024008 18691 catalog_manager.cc:4841] TS 6d24058590b9402ba68586e937173af4 (127.6.138.66:44237): tablet 882c3ac3ee23472280531a7cc6e95121 (table test-table [id=4b965028027b432093b789f31ed186a2]) successfully deleted
I20260807 18:38:46.024010 18692 catalog_manager.cc:4841] TS 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059): tablet 882c3ac3ee23472280531a7cc6e95121 (table test-table [id=4b965028027b432093b789f31ed186a2]) successfully deleted
I20260807 18:38:46.025141 19498 ts_tablet_manager.cc:1918] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:46.025235 19498 log.cc:1196] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/882c3ac3ee23472280531a7cc6e95121
I20260807 18:38:46.025488 19498 ts_tablet_manager.cc:1939] T 882c3ac3ee23472280531a7cc6e95121 P 173662d55dd946608a6e0ffd293a2bb4: Deleting consensus metadata
I20260807 18:38:46.026288 18695 catalog_manager.cc:4841] TS 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273): tablet 882c3ac3ee23472280531a7cc6e95121 (table test-table [id=4b965028027b432093b789f31ed186a2]) successfully deleted
Aug 07 18:38:46 dist-test-slave-2d19 krb5kdc[18651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:38:46 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:46 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:38:46.057569 18783 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:46.049173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41566 (local address 127.6.138.126:38003)
0807 18:38:46.049307 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:46.049312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:46.049614 (+ 302us) server_negotiation.cc:407] Connection header received
0807 18:38:46.049754 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:46.049758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:46.049826 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:46.049937 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:46.050911 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.051781 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.053077 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.053462 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.053576 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:46.057431 (+ 3855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41566: BlockingRecv error: recv got EOF from 127.0.0.1:41566 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 18:38:46 dist-test-slave-2d19 krb5kdc[18651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM
I20260807 18:38:46.070688 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:46.059628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.6.138.126:38003)
0807 18:38:46.059731 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:46.059735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:46.059946 (+ 211us) server_negotiation.cc:407] Connection header received
0807 18:38:46.060072 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:46.060075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:46.060121 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:46.060207 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:46.061001 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.061787 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.063031 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.063368 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.063476 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:46.066574 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:46.066578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:46.066582 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:46.066612 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:46.068584 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:46.069091 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:46.069095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:46.069097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:46.069199 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:46.069506 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:46.069512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:46.069515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:46.069767 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:46.069915 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:46.070208 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 18:38:46.070377 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 18:38:46.074239 18709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:38:46.074771 18709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:46.079912 18914 tablet_service.cc:1467] Processing CreateTablet for tablet 8968c45e946e4c968ce41bc2ab54631e (DEFAULT_TABLE table=test-table [id=e0467b806001422786a21d7872f431bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:46.080075 19110 tablet_service.cc:1467] Processing CreateTablet for tablet 8968c45e946e4c968ce41bc2ab54631e (DEFAULT_TABLE table=test-table [id=e0467b806001422786a21d7872f431bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:46.080384 18914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8968c45e946e4c968ce41bc2ab54631e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:46.080425 19110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8968c45e946e4c968ce41bc2ab54631e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:46.083005 19465 tablet_bootstrap.cc:492] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: Bootstrap starting.
I20260807 18:38:46.083766 19310 tablet_service.cc:1467] Processing CreateTablet for tablet 8968c45e946e4c968ce41bc2ab54631e (DEFAULT_TABLE table=test-table [id=e0467b806001422786a21d7872f431bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:46.083923 19465 tablet_bootstrap.cc:654] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:46.084066 19310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8968c45e946e4c968ce41bc2ab54631e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:46.085345 19465 tablet_bootstrap.cc:492] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: No bootstrap required, opened a new log
I20260807 18:38:46.085376 19464 tablet_bootstrap.cc:492] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: Bootstrap starting.
I20260807 18:38:46.085434 19465 ts_tablet_manager.cc:1397] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 18:38:46.085850 19465 raft_consensus.cc:348] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.085968 19465 raft_consensus.cc:374] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:46.086001 19465 raft_consensus.cc:729] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d24058590b9402ba68586e937173af4, State: Initialized, Role: FOLLOWER
I20260807 18:38:46.086128 19465 consensus_queue.cc:260] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.086344 19464 tablet_bootstrap.cc:654] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:46.086414 19465 ts_tablet_manager.cc:1428] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 18:38:46.087188 19463 tablet_bootstrap.cc:492] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: Bootstrap starting.
I20260807 18:38:46.088056 19464 tablet_bootstrap.cc:492] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: No bootstrap required, opened a new log
I20260807 18:38:46.088141 19464 ts_tablet_manager.cc:1397] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 18:38:46.088527 19463 tablet_bootstrap.cc:654] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:46.088728 19464 raft_consensus.cc:348] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.088855 19464 raft_consensus.cc:374] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:46.088908 19464 raft_consensus.cc:729] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173662d55dd946608a6e0ffd293a2bb4, State: Initialized, Role: FOLLOWER
I20260807 18:38:46.089056 19464 consensus_queue.cc:260] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.089380 19464 ts_tablet_manager.cc:1428] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:46.090266 19463 tablet_bootstrap.cc:492] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: No bootstrap required, opened a new log
I20260807 18:38:46.090361 19463 ts_tablet_manager.cc:1397] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:38:46.090803 19463 raft_consensus.cc:348] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.090936 19463 raft_consensus.cc:374] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:46.090969 19463 raft_consensus.cc:729] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8afcf0879347a0b32934c8268b1fd0, State: Initialized, Role: FOLLOWER
I20260807 18:38:46.091094 19463 consensus_queue.cc:260] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.092017 19463 ts_tablet_manager.cc:1428] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 18:38:46.212126 19490 raft_consensus.cc:482] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:46.212338 19490 raft_consensus.cc:504] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.212844 19490 leader_election.cc:288] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273), 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059)
I20260807 18:38:46.213296 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8968c45e946e4c968ce41bc2ab54631e" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8afcf0879347a0b32934c8268b1fd0" is_pre_election: true
I20260807 18:38:46.213318 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8968c45e946e4c968ce41bc2ab54631e" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173662d55dd946608a6e0ffd293a2bb4" is_pre_election: true
I20260807 18:38:46.213456 19344 raft_consensus.cc:2454] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 0.
I20260807 18:38:46.213501 18939 raft_consensus.cc:2454] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 0.
I20260807 18:38:46.213797 19047 leader_election.cc:302] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d24058590b9402ba68586e937173af4, 6f8afcf0879347a0b32934c8268b1fd0; no voters:
I20260807 18:38:46.213994 19490 raft_consensus.cc:2781] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:46.214082 19490 raft_consensus.cc:482] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:46.214138 19490 raft_consensus.cc:3037] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:46.215353 19490 raft_consensus.cc:504] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.215778 19490 leader_election.cc:288] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 election: Requested vote from peers 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273), 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059)
I20260807 18:38:46.216122 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8968c45e946e4c968ce41bc2ab54631e" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173662d55dd946608a6e0ffd293a2bb4"
I20260807 18:38:46.216146 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8968c45e946e4c968ce41bc2ab54631e" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8afcf0879347a0b32934c8268b1fd0"
I20260807 18:38:46.216271 18939 raft_consensus.cc:3037] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:46.216310 19344 raft_consensus.cc:3037] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:46.217397 19344 raft_consensus.cc:2454] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 1.
I20260807 18:38:46.217422 18939 raft_consensus.cc:2454] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 1.
I20260807 18:38:46.217696 19047 leader_election.cc:302] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d24058590b9402ba68586e937173af4, 6f8afcf0879347a0b32934c8268b1fd0; no voters:
I20260807 18:38:46.217868 19490 raft_consensus.cc:2781] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:46.218092 19490 raft_consensus.cc:686] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 1 LEADER]: Becoming Leader. State: Replica: 6d24058590b9402ba68586e937173af4, State: Running, Role: LEADER
I20260807 18:38:46.218252 19490 consensus_queue.cc:237] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.219884 18708 catalog_manager.cc:5495] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d24058590b9402ba68586e937173af4 (127.6.138.66). New cstate: current_term: 1 leader_uuid: "6d24058590b9402ba68586e937173af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } health_report { overall_health: UNKNOWN } } }
I20260807 18:38:46.249225 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:46.245787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.6.138.66:44237)
0807 18:38:46.245924 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:46.245928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:46.245942 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:38:46.246082 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:46.246084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:46.246143 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:46.246233 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:46.246573 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.247087 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.247975 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.248221 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.248354 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:46.248776 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:46.248848 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:46.248902 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:46.248988 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 18:38:46.249059 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:38:46.251302 19344 raft_consensus.cc:1261] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Refusing update from remote peer 6d24058590b9402ba68586e937173af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:46.251317 18939 raft_consensus.cc:1261] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Refusing update from remote peer 6d24058590b9402ba68586e937173af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:46.251783 19490 consensus_queue.cc:1035] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:46.251974 19579 consensus_queue.cc:1035] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:46.259259 18708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:46.259712 18708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:46.261752 18708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80: Sending DeleteTablet for 3 replicas of tablet 8968c45e946e4c968ce41bc2ab54631e
I20260807 18:38:46.262348 19110 tablet_service.cc:1514] Processing DeleteTablet for tablet 8968c45e946e4c968ce41bc2ab54631e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:46 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:59828
I20260807 18:38:46.262398 19310 tablet_service.cc:1514] Processing DeleteTablet for tablet 8968c45e946e4c968ce41bc2ab54631e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:46 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45000
I20260807 18:38:46.262408 18914 tablet_service.cc:1514] Processing DeleteTablet for tablet 8968c45e946e4c968ce41bc2ab54631e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:46 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:60112
I20260807 18:38:46.262562 19496 tablet_replica.cc:330] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: stopping tablet replica
I20260807 18:38:46.262614 19499 tablet_replica.cc:330] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: stopping tablet replica
I20260807 18:38:46.262687 19496 raft_consensus.cc:2229] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:46.262738 19499 raft_consensus.cc:2229] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:46.262820 19496 raft_consensus.cc:2258] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:46.262928 19499 raft_consensus.cc:2258] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:46.263127 19498 tablet_replica.cc:330] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: stopping tablet replica
I20260807 18:38:46.263090 18708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:38:46.263268 19498 raft_consensus.cc:2229] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:46.263398 19498 raft_consensus.cc:2258] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 18:38:46.263451 18708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:38:46.263659 19499 ts_tablet_manager.cc:1905] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:46.264384 19498 ts_tablet_manager.cc:1905] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:46.264580 19496 ts_tablet_manager.cc:1905] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:46.266414 19499 ts_tablet_manager.cc:1918] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:46.266484 19499 log.cc:1196] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/8968c45e946e4c968ce41bc2ab54631e
I20260807 18:38:46.266601 19496 ts_tablet_manager.cc:1918] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:46.266659 19496 log.cc:1196] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/8968c45e946e4c968ce41bc2ab54631e
I20260807 18:38:46.266749 19499 ts_tablet_manager.cc:1939] T 8968c45e946e4c968ce41bc2ab54631e P 6d24058590b9402ba68586e937173af4: Deleting consensus metadata
I20260807 18:38:46.266860 19496 ts_tablet_manager.cc:1939] T 8968c45e946e4c968ce41bc2ab54631e P 6f8afcf0879347a0b32934c8268b1fd0: Deleting consensus metadata
I20260807 18:38:46.267148 19498 ts_tablet_manager.cc:1918] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:38:46.267218 19498 log.cc:1196] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/8968c45e946e4c968ce41bc2ab54631e
I20260807 18:38:46.267515 18691 catalog_manager.cc:4841] TS 6d24058590b9402ba68586e937173af4 (127.6.138.66:44237): tablet 8968c45e946e4c968ce41bc2ab54631e (table test-table [id=e0467b806001422786a21d7872f431bf]) successfully deleted
I20260807 18:38:46.267751 18692 catalog_manager.cc:4841] TS 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059): tablet 8968c45e946e4c968ce41bc2ab54631e (table test-table [id=e0467b806001422786a21d7872f431bf]) successfully deleted
I20260807 18:38:46.267474 19498 ts_tablet_manager.cc:1939] T 8968c45e946e4c968ce41bc2ab54631e P 173662d55dd946608a6e0ffd293a2bb4: Deleting consensus metadata
I20260807 18:38:46.268671 18695 catalog_manager.cc:4841] TS 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273): tablet 8968c45e946e4c968ce41bc2ab54631e (table test-table [id=e0467b806001422786a21d7872f431bf]) successfully deleted
I20260807 18:38:46.270589 18914 tablet_service.cc:1467] Processing CreateTablet for tablet c2ee8e6084814ebd8b064bfcbae3dd8d (DEFAULT_TABLE table=test-table [id=1061a4dfde4947f894a3d28754af755a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:46.270591 19110 tablet_service.cc:1467] Processing CreateTablet for tablet c2ee8e6084814ebd8b064bfcbae3dd8d (DEFAULT_TABLE table=test-table [id=1061a4dfde4947f894a3d28754af755a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:46.270659 19310 tablet_service.cc:1467] Processing CreateTablet for tablet c2ee8e6084814ebd8b064bfcbae3dd8d (DEFAULT_TABLE table=test-table [id=1061a4dfde4947f894a3d28754af755a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:38:46.270860 19110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ee8e6084814ebd8b064bfcbae3dd8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:46.270860 18914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ee8e6084814ebd8b064bfcbae3dd8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:46.270974 19310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ee8e6084814ebd8b064bfcbae3dd8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:46.273206 19463 tablet_bootstrap.cc:492] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: Bootstrap starting.
I20260807 18:38:46.273386 19464 tablet_bootstrap.cc:492] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4: Bootstrap starting.
I20260807 18:38:46.273648 19465 tablet_bootstrap.cc:492] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: Bootstrap starting.
I20260807 18:38:46.274108 19463 tablet_bootstrap.cc:654] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:46.274266 19464 tablet_bootstrap.cc:654] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:46.274432 19465 tablet_bootstrap.cc:654] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:46.275569 19465 tablet_bootstrap.cc:492] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: No bootstrap required, opened a new log
I20260807 18:38:46.275672 19465 ts_tablet_manager.cc:1397] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 18:38:46.275674 19464 tablet_bootstrap.cc:492] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4: No bootstrap required, opened a new log
I20260807 18:38:46.275760 19464 ts_tablet_manager.cc:1397] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:38:46.276085 19465 raft_consensus.cc:348] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.276213 19465 raft_consensus.cc:374] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:46.276156 19464 raft_consensus.cc:348] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.276238 19465 raft_consensus.cc:729] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d24058590b9402ba68586e937173af4, State: Initialized, Role: FOLLOWER
I20260807 18:38:46.276432 19465 consensus_queue.cc:260] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.276710 19465 ts_tablet_manager.cc:1428] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:38:46.276746 19463 tablet_bootstrap.cc:492] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: No bootstrap required, opened a new log
I20260807 18:38:46.276829 19463 ts_tablet_manager.cc:1397] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260807 18:38:46.277263 19463 raft_consensus.cc:348] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.277393 19463 raft_consensus.cc:374] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:46.277426 19463 raft_consensus.cc:729] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8afcf0879347a0b32934c8268b1fd0, State: Initialized, Role: FOLLOWER
I20260807 18:38:46.277552 19463 consensus_queue.cc:260] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.277848 19463 ts_tablet_manager.cc:1428] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260807 18:38:46.278389 19464 raft_consensus.cc:374] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:46.278455 19464 raft_consensus.cc:729] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173662d55dd946608a6e0ffd293a2bb4, State: Initialized, Role: FOLLOWER
I20260807 18:38:46.278577 19464 consensus_queue.cc:260] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.278883 19464 ts_tablet_manager.cc:1428] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260807 18:38:46.440296 19589 raft_consensus.cc:482] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:38:46.440595 19589 raft_consensus.cc:504] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.441155 19589 leader_election.cc:288] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273), 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059)
I20260807 18:38:46.441682 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ee8e6084814ebd8b064bfcbae3dd8d" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8afcf0879347a0b32934c8268b1fd0" is_pre_election: true
I20260807 18:38:46.441699 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ee8e6084814ebd8b064bfcbae3dd8d" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173662d55dd946608a6e0ffd293a2bb4" is_pre_election: true
I20260807 18:38:46.441825 19344 raft_consensus.cc:2454] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 0.
I20260807 18:38:46.441968 18939 raft_consensus.cc:2454] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 0.
I20260807 18:38:46.442217 19047 leader_election.cc:302] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d24058590b9402ba68586e937173af4, 6f8afcf0879347a0b32934c8268b1fd0; no voters:
I20260807 18:38:46.442425 19589 raft_consensus.cc:2781] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:38:46.442484 19589 raft_consensus.cc:482] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:38:46.442512 19589 raft_consensus.cc:3037] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:46.443784 19589 raft_consensus.cc:504] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.444177 19589 leader_election.cc:288] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 election: Requested vote from peers 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273), 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059)
I20260807 18:38:46.444648 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ee8e6084814ebd8b064bfcbae3dd8d" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173662d55dd946608a6e0ffd293a2bb4"
I20260807 18:38:46.444684 19344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ee8e6084814ebd8b064bfcbae3dd8d" candidate_uuid: "6d24058590b9402ba68586e937173af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f8afcf0879347a0b32934c8268b1fd0"
I20260807 18:38:46.444787 18939 raft_consensus.cc:3037] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:46.444840 19344 raft_consensus.cc:3037] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:46.445858 19344 raft_consensus.cc:2454] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 1.
I20260807 18:38:46.445875 18939 raft_consensus.cc:2454] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d24058590b9402ba68586e937173af4 in term 1.
I20260807 18:38:46.446189 19047 leader_election.cc:302] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d24058590b9402ba68586e937173af4, 6f8afcf0879347a0b32934c8268b1fd0; no voters:
I20260807 18:38:46.446380 19589 raft_consensus.cc:2781] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:46.446497 19589 raft_consensus.cc:686] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 1 LEADER]: Becoming Leader. State: Replica: 6d24058590b9402ba68586e937173af4, State: Running, Role: LEADER
I20260807 18:38:46.446657 19589 consensus_queue.cc:237] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } }
I20260807 18:38:46.448570 18708 catalog_manager.cc:5495] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d24058590b9402ba68586e937173af4 (127.6.138.66). New cstate: current_term: 1 leader_uuid: "6d24058590b9402ba68586e937173af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d24058590b9402ba68586e937173af4" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 44237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 } health_report { overall_health: UNKNOWN } } }
I20260807 18:38:46.481070 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:46.477493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37273 (local address 127.6.138.66:59105)
0807 18:38:46.477867 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:46.477883 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:46.478021 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:46.478306 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:46.478309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:46.478316 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:46.478425 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:46.478436 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.479180 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.479184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:46.479817 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:46.479826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.480326 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.480330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:46.480419 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:46.480469 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:46.480772 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:46.480782 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:38:46.480848 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 18:38:46.480930 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":125,"threads_started":1}
I20260807 18:38:46.481070 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:46.477643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:59105 (local address 127.6.138.65:37273)
0807 18:38:46.477875 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:46.477882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:46.477960 (+ 78us) server_negotiation.cc:407] Connection header received
0807 18:38:46.478153 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:46.478156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:46.478201 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:46.478284 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:46.478536 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.479070 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.479941 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.480189 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.480327 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:46.480616 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:46.480688 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:46.480768 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:46.480858 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 18:38:46.480911 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":66,"threads_started":1}
I20260807 18:38:46.486830 19344 raft_consensus.cc:1261] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Refusing update from remote peer 6d24058590b9402ba68586e937173af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:46.486871 18939 raft_consensus.cc:1261] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Refusing update from remote peer 6d24058590b9402ba68586e937173af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:38:46.487283 19589 consensus_queue.cc:1035] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f8afcf0879347a0b32934c8268b1fd0" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:46.487447 19490 consensus_queue.cc:1035] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "173662d55dd946608a6e0ffd293a2bb4" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:38:46.503902 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:46.500473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:56805 (local address 127.6.138.66:44237)
0807 18:38:46.500601 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:46.500606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:46.500813 (+ 207us) server_negotiation.cc:407] Connection header received
0807 18:38:46.500949 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:46.500952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:46.501002 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:46.501083 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:38:46.501360 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.501869 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.502696 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.503011 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.503111 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:46.503420 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:38:46.503503 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:38:46.503572 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:46.503706 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 18:38:46.503762 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 18:38:46.503897 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:46.500412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:44237 (local address 127.6.138.65:56805)
0807 18:38:46.500738 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:46.500753 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:46.500856 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:46.501102 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:46.501107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:46.501121 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:38:46.501243 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:46.501249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.501960 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.501962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:46.502581 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:46.502589 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:46.503144 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:46.503167 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:46.503253 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:46.503288 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:38:46.503598 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:38:46.503605 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:38:46.503671 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 18:38:46.503740 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":79,"threads_started":1}
I20260807 18:38:46.522755 18704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:46.522904 18704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:38:46.524762 18704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea7f5ff830824d7481f27f37986e0d80: Sending DeleteTablet for 3 replicas of tablet c2ee8e6084814ebd8b064bfcbae3dd8d
I20260807 18:38:46.525288 19110 tablet_service.cc:1514] Processing DeleteTablet for tablet c2ee8e6084814ebd8b064bfcbae3dd8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:46 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:59828
I20260807 18:38:46.525288 18914 tablet_service.cc:1514] Processing DeleteTablet for tablet c2ee8e6084814ebd8b064bfcbae3dd8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:46 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:60112
I20260807 18:38:46.525352 19310 tablet_service.cc:1514] Processing DeleteTablet for tablet c2ee8e6084814ebd8b064bfcbae3dd8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:38:46 UTC) from {username='oryx', principal='oryx/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45000
I20260807 18:38:46.525487 19499 tablet_replica.cc:330] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: stopping tablet replica
I20260807 18:38:46.525506 19498 tablet_replica.cc:330] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4: stopping tablet replica
I20260807 18:38:46.525542 19496 tablet_replica.cc:330] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: stopping tablet replica
I20260807 18:38:46.525611 19499 raft_consensus.cc:2229] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:38:46.525624 19498 raft_consensus.cc:2229] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:46.525668 19496 raft_consensus.cc:2229] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:38:46.525730 19498 raft_consensus.cc:2258] T c2ee8e6084814ebd8b064bfcbae3dd8d P 173662d55dd946608a6e0ffd293a2bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:46.525772 19496 raft_consensus.cc:2258] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:46.525847 19499 raft_consensus.cc:2258] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:38:46.526156 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 18813
I20260807 18:38:46.526643 19499 ts_tablet_manager.cc:1905] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:46.526741 19496 ts_tablet_manager.cc:1905] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:38:46.529080 19499 ts_tablet_manager.cc:1918] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:46.529181 19499 log.cc:1196] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/c2ee8e6084814ebd8b064bfcbae3dd8d
I20260807 18:38:46.529456 19499 ts_tablet_manager.cc:1939] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6d24058590b9402ba68586e937173af4: Deleting consensus metadata
I20260807 18:38:46.530229 19496 ts_tablet_manager.cc:1918] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:38:46.530319 19496 log.cc:1196] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/c2ee8e6084814ebd8b064bfcbae3dd8d
I20260807 18:38:46.530362 18691 catalog_manager.cc:4841] TS 6d24058590b9402ba68586e937173af4 (127.6.138.66:44237): tablet c2ee8e6084814ebd8b064bfcbae3dd8d (table test-table [id=1061a4dfde4947f894a3d28754af755a]) successfully deleted
I20260807 18:38:46.530552 19496 ts_tablet_manager.cc:1939] T c2ee8e6084814ebd8b064bfcbae3dd8d P 6f8afcf0879347a0b32934c8268b1fd0: Deleting consensus metadata
I20260807 18:38:46.532529 18692 catalog_manager.cc:4841] TS 6f8afcf0879347a0b32934c8268b1fd0 (127.6.138.67:43059): tablet c2ee8e6084814ebd8b064bfcbae3dd8d (table test-table [id=1061a4dfde4947f894a3d28754af755a]) successfully deleted
W20260807 18:38:46.543797 18695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:37273 (error 108)
I20260807 18:38:46.544139 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 19019
I20260807 18:38:46.551379 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:46.549974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37273 (local address 127.0.0.1:41866)
0807 18:38:46.551134 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:46.551234 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:37273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1050,"thread_start_us":662,"threads_started":1}
W20260807 18:38:46.551584 18695 catalog_manager.cc:4584] TS 173662d55dd946608a6e0ffd293a2bb4 (127.6.138.65:37273): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2ee8e6084814ebd8b064bfcbae3dd8d: Network error: Client connection negotiation failed: client connection to 127.6.138.65:37273: connect: Connection refused (error 111)
I20260807 18:38:46.558342 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 19217
I20260807 18:38:46.572465 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 18667
2026-08-07T18:38:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3795 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:46 dist-test-slave-2d19 krb5kdc[19621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:46 dist-test-slave-2d19 krb5kdc[19621](info): set up 2 sockets
Aug 07 18:38:46 dist-test-slave-2d19 krb5kdc[19621](info): commencing operation
krb5kdc: starting...
W20260807 18:38:47.848068 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:47 dist-test-slave-2d19 krb5kdc[19621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:47.984236 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:35823
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:35823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:48.100018 19645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:48.100390 19645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:48.100471 19645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:48.104595 19645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:48.104692 19645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:48.104732 19645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:48.104768 19645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:48.104801 19645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:48.110337 19645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:35823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:35823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:48.111557 19645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:48.112699 19645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:48.121533 19659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:48.121513 19656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:48.121490 19657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:48.121949 19645 server_base.cc:1034] running on GCE node
I20260807 18:38:48.122584 19645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:48.123669 19645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:48.124827 19645 hybrid_clock.cc:648] HybridClock initialized: now 1786127928124807 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T18:38:48Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1549 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:38:48 dist-test-slave-2d19 krb5kdc[19667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:38:48 dist-test-slave-2d19 krb5kdc[19667](info): set up 2 sockets
Aug 07 18:38:48 dist-test-slave-2d19 krb5kdc[19667](info): commencing operation
krb5kdc: starting...
W20260807 18:38:49.447454 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:38:49 dist-test-slave-2d19 krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:38:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:38:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:38:49.588213 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:44969
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:44969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:49.705871 19685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:49.706210 19685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:49.706290 19685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:49.710382 19685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:49.710477 19685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:49.710510 19685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:49.710567 19685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:49.710601 19685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:49.716197 19685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:44969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:44969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:49.717406 19685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:49.718499 19685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:49.727324 19696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:49.727377 19685 server_base.cc:1034] running on GCE node
W20260807 18:38:49.727336 19693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:49.727324 19694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:49.728231 19685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:49.729480 19685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:49.730680 19685 hybrid_clock.cc:648] HybridClock initialized: now 1786127929730666 us; error 30 us; skew 500 ppm
Aug 07 18:38:49 dist-test-slave-2d19 krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127929, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:49.737986 19685 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:49.739151 19685 webserver.cc:460] Webserver started at http://127.6.138.126:36737/ using document root <none> and password file <none>
I20260807 18:38:49.739759 19685 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:49.739854 19685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:49.740099 19685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:49.742283 19685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "1154c74af65d41a695cb50cd093a6e14"
format_stamp: "Formatted at 2026-08-07 18:38:49 on dist-test-slave-2d19"
server_key: "00e1c8010a32c95448736d51228a5323"
server_key_iv: "b783782a3db2da5db6f087d11bf58303"
server_key_version: "encryptionkey@0"
I20260807 18:38:49.742866 19685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "1154c74af65d41a695cb50cd093a6e14"
format_stamp: "Formatted at 2026-08-07 18:38:49 on dist-test-slave-2d19"
server_key: "00e1c8010a32c95448736d51228a5323"
server_key_iv: "b783782a3db2da5db6f087d11bf58303"
server_key_version: "encryptionkey@0"
I20260807 18:38:49.746795 19685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:49.749642 19714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:49.751467 19685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:38:49.751645 19685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "1154c74af65d41a695cb50cd093a6e14"
format_stamp: "Formatted at 2026-08-07 18:38:49 on dist-test-slave-2d19"
server_key: "00e1c8010a32c95448736d51228a5323"
server_key_iv: "b783782a3db2da5db6f087d11bf58303"
server_key_version: "encryptionkey@0"
I20260807 18:38:49.751773 19685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:49.764233 19685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:49.768153 19685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:49.768411 19685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:49.776126 19685 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:44969
I20260807 18:38:49.776152 19801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:44969 every 8 connection(s)
I20260807 18:38:49.777194 19685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:49.780128 19803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:38:49.786104 19803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: Bootstrap starting.
I20260807 18:38:49.786320 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 19685
I20260807 18:38:49.786475 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:38:49.786772 6697 external_mini_cluster.cc:1428] Setting key 2acbe22b2018e37e6259477b08a07909
I20260807 18:38:49.789191 19803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: Neither blocks nor log segments found. Creating new log.
I20260807 18:38:49.790148 19803 log.cc:824] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:49.792706 19803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: No bootstrap required, opened a new log
I20260807 18:38:49.796809 19803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } }
Aug 07 18:38:49 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:49.797073 19803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:38:49.797171 19803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1154c74af65d41a695cb50cd093a6e14, State: Initialized, Role: FOLLOWER
I20260807 18:38:49.797704 19803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } }
I20260807 18:38:49.797845 19803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:49.797945 19803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:49.798097 19803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:38:49.799054 19803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } }
I20260807 18:38:49.799398 19803 leader_election.cc:302] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1154c74af65d41a695cb50cd093a6e14; no voters:
I20260807 18:38:49.799715 19803 leader_election.cc:288] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:38:49.799846 19814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:38:49.800079 19814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 1 LEADER]: Becoming Leader. State: Replica: 1154c74af65d41a695cb50cd093a6e14, State: Running, Role: LEADER
I20260807 18:38:49.800556 19814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } }
I20260807 18:38:49.801206 19803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:49.802332 19815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1154c74af65d41a695cb50cd093a6e14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } } }
I20260807 18:38:49.802474 19815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:49.802717 19815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1154c74af65d41a695cb50cd093a6e14. Latest consensus state: current_term: 1 leader_uuid: "1154c74af65d41a695cb50cd093a6e14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } } }
I20260807 18:38:49.802807 19815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:49.804445 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:49.788342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46544 (local address 127.6.138.126:44969)
0807 18:38:49.788933 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:49.788959 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:49.789005 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:38:49.789937 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:49.789964 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:49.790411 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:49.790789 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:49.791706 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:49.793206 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:49.794104 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:49.794825 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:49.794992 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:49.797728 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:49.797733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:49.797747 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:49.797776 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:49.800013 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:49.800629 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:49.800634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:49.800641 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:49.800786 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:49.801131 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:49.801134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:49.801136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:49.801673 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:49.802659 (+ 986us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:49.802825 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 18:38:49.803154 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":137,"threads_started":1}
W20260807 18:38:49.807173 19837 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:38:49.807255 19837 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:38:49.807320 19828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:49.808205 19828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:49.814042 19828 catalog_manager.cc:1347] Generated new cluster ID: 6f89b8fa503d4621a3dab22d25533700
I20260807 18:38:49.814122 19828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:49.847533 19828 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:38:49.848987 19828 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:49.856348 19828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: Generated new TSK 0
I20260807 18:38:49.857049 19828 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:38:49.894922 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44969
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:50.016870 19844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:50.017140 19844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:50.017184 19844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:50.021206 19844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:50.021266 19844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:50.021323 19844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:50.026593 19844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:50.027638 19844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:50.028699 19844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:50.038697 19858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:50.038705 19855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:50.038738 19850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:50.038759 19844 server_base.cc:1034] running on GCE node
I20260807 18:38:50.039328 19844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:50.039963 19844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:50.041131 19844 hybrid_clock.cc:648] HybridClock initialized: now 1786127930041089 us; error 53 us; skew 500 ppm
Aug 07 18:38:50 dist-test-slave-2d19 krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127930, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:50.044090 19844 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:38:50.044989 19844 webserver.cc:460] Webserver started at http://127.6.138.65:44433/ using document root <none> and password file <none>
I20260807 18:38:50.045617 19844 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:50.045683 19844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:50.045935 19844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:50.048019 19844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "583c73637f0c4da0bc02a57f5f43eafd"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "20f340ba3275c5b470c9b3ea6fe688f1"
server_key_iv: "70795f71fe20a3d2e7e33e48e2ec12ea"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.048705 19844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "583c73637f0c4da0bc02a57f5f43eafd"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "20f340ba3275c5b470c9b3ea6fe688f1"
server_key_iv: "70795f71fe20a3d2e7e33e48e2ec12ea"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.052582 19844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:38:50.055053 19869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.056567 19844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:50.056699 19844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "583c73637f0c4da0bc02a57f5f43eafd"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "20f340ba3275c5b470c9b3ea6fe688f1"
server_key_iv: "70795f71fe20a3d2e7e33e48e2ec12ea"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.056826 19844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:50.080847 19844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:50.084432 19844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:50.084648 19844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:50.085296 19844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:50.086248 19844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:50.086321 19844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.086392 19844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:50.086443 19844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.095835 19844 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:42429
I20260807 18:38:50.095868 20036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:42429 every 8 connection(s)
I20260807 18:38:50.097131 19844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:50.103722 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 19844
I20260807 18:38:50.103870 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:38:50.104172 6697 external_mini_cluster.cc:1428] Setting key 0ad96a90185fef9e5ae399c045cca2db
Aug 07 18:38:50 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127930, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:50.112867 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:50.099216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:60299 (local address 127.6.138.126:44969)
0807 18:38:50.099352 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:50.099357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:50.100422 (+ 1065us) server_negotiation.cc:407] Connection header received
0807 18:38:50.101371 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:50.101375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:50.101448 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:50.101527 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:50.102698 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.103245 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.104565 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.104816 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.104930 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:50.107633 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:50.107636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:50.107639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:50.107667 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:50.109711 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:50.110475 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:50.110480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:50.110483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:50.110613 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:50.111128 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:50.111132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:50.111134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:50.111416 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:50.111510 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:50.112387 (+ 877us) server_negotiation.cc:299] Negotiation successful
0807 18:38:50.112616 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:38:50.113876 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:50.099471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:60299)
0807 18:38:50.100201 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:50.100244 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:50.101139 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:50.101704 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:50.101711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:50.102074 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:50.102552 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:50.102565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.103373 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.103376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:50.104441 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:50.104451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.104980 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.104984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:50.105163 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:50.105769 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:50.105786 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:50.107466 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:50.109874 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:50.109883 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:50.109902 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:50.110340 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:50.110761 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:50.110767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:50.110772 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:50.110978 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:50.111626 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:50.111632 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:50.112020 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 18:38:50.112316 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 18:38:50.112691 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":123,"threads_started":1}
I20260807 18:38:50.117506 20037 heartbeater.cc:347] Connected to a master server at 127.6.138.126:44969
I20260807 18:38:50.117844 20037 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:38:50.118561 20037 heartbeater.cc:511] Master 127.6.138.126:44969 requested a full tablet report, sending...
I20260807 18:38:50.120410 19744 ts_manager.cc:194] Registered new tserver with Master: 583c73637f0c4da0bc02a57f5f43eafd (127.6.138.65:42429)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
I20260807 18:38:50.122035 19744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:60299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:38:50.146687 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44969
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:50.274055 20044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:50.274315 20044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:50.274358 20044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:50.278329 20044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:50.278388 20044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:50.278445 20044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:50.283818 20044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:50.284885 20044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:50.285949 20044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:50.296108 20059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:50.296082 20054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:50.296101 20053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:50.296434 20044 server_base.cc:1034] running on GCE node
I20260807 18:38:50.296970 20044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:50.297668 20044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:50.298833 20044 hybrid_clock.cc:648] HybridClock initialized: now 1786127930298826 us; error 79 us; skew 500 ppm
Aug 07 18:38:50 dist-test-slave-2d19 krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127930, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:50.301926 20044 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:38:50.302994 20044 webserver.cc:460] Webserver started at http://127.6.138.66:37845/ using document root <none> and password file <none>
I20260807 18:38:50.303736 20044 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:50.303810 20044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:50.304059 20044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:50.306248 20044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "dcefddf26f384f8caaaea99abd6cf9d7"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "beb890cf615371e8b21441b376fe4c94"
server_key_iv: "0069c337cdf827853bd913d06884f47e"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.306826 20044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "dcefddf26f384f8caaaea99abd6cf9d7"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "beb890cf615371e8b21441b376fe4c94"
server_key_iv: "0069c337cdf827853bd913d06884f47e"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.310659 20044 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:38:50.313195 20075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.314411 20044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:50.314543 20044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "dcefddf26f384f8caaaea99abd6cf9d7"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "beb890cf615371e8b21441b376fe4c94"
server_key_iv: "0069c337cdf827853bd913d06884f47e"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.314646 20044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:50.335729 20044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:50.338937 20044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:50.339164 20044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:50.339789 20044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:50.340735 20044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:50.340787 20044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.340852 20044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:50.340888 20044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.350661 20044 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:39545
I20260807 18:38:50.350685 20241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:39545 every 8 connection(s)
I20260807 18:38:50.351739 20044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:50.356529 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 20044
I20260807 18:38:50.356664 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:38:50.356973 6697 external_mini_cluster.cc:1428] Setting key 9492bae54b795bc2983e6b995cd466be
Aug 07 18:38:50 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127930, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:50.367925 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:50.353777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:55097 (local address 127.6.138.126:44969)
0807 18:38:50.353921 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:50.353925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:50.354851 (+ 926us) server_negotiation.cc:407] Connection header received
0807 18:38:50.356396 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:50.356399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:50.356485 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:50.356607 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:50.358154 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.358770 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.360706 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.361117 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.361242 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:50.364048 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:50.364052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:50.364055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:50.364090 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:50.365659 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:50.366238 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:50.366242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:50.366243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:50.366334 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:50.366698 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:50.366703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:50.366704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:50.366904 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:50.367016 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:50.367608 (+ 592us) server_negotiation.cc:299] Negotiation successful
0807 18:38:50.367753 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:38:50.368930 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:50.354223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:55097)
0807 18:38:50.354695 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:50.354741 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:50.356067 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:50.356823 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:50.356835 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:50.357371 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:50.358002 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:50.358026 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.358919 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.358925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:50.360558 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:50.360576 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.361289 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.361292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:50.361468 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:50.362132 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:50.362166 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:50.363868 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:50.365790 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:50.365796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:50.365814 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:50.366118 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:50.366431 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:50.366434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:50.366436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:50.366596 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:50.367018 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:50.367028 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:50.367347 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 18:38:50.367556 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 18:38:50.367945 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":117,"threads_started":1}
I20260807 18:38:50.370182 20244 heartbeater.cc:347] Connected to a master server at 127.6.138.126:44969
I20260807 18:38:50.370540 20244 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
I20260807 18:38:50.371192 20244 heartbeater.cc:511] Master 127.6.138.126:44969 requested a full tablet report, sending...
I20260807 18:38:50.372501 19744 ts_manager.cc:194] Registered new tserver with Master: dcefddf26f384f8caaaea99abd6cf9d7 (127.6.138.66:39545)
I20260807 18:38:50.373392 19744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:55097
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:38:50.391557 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44969
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:50.514823 20251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:50.515141 20251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:50.515214 20251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:50.519474 20251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:50.519551 20251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:50.519635 20251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:50.525223 20251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:44969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:50.526347 20251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:50.527421 20251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:50.537905 20259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:50.538169 20256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:50.537905 20257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:50.537959 20251 server_base.cc:1034] running on GCE node
I20260807 18:38:50.538671 20251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:50.539330 20251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:50.540505 20251 hybrid_clock.cc:648] HybridClock initialized: now 1786127930540483 us; error 33 us; skew 500 ppm
Aug 07 18:38:50 dist-test-slave-2d19 krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127930, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:50.543315 20251 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:38:50.544245 20251 webserver.cc:460] Webserver started at http://127.6.138.67:36517/ using document root <none> and password file <none>
I20260807 18:38:50.544871 20251 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:50.544961 20251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:50.545199 20251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:38:50.547281 20251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "8b8ceb9907324b9cb31e1e83243ddd10"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "4a597686d003ccabe9d030341059970f"
server_key_iv: "37b33238d77952674939edc90bee35b6"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.547865 20251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "8b8ceb9907324b9cb31e1e83243ddd10"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "4a597686d003ccabe9d030341059970f"
server_key_iv: "37b33238d77952674939edc90bee35b6"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.551788 20251 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:38:50.554565 20267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.556027 20251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:38:50.556161 20251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "8b8ceb9907324b9cb31e1e83243ddd10"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "4a597686d003ccabe9d030341059970f"
server_key_iv: "37b33238d77952674939edc90bee35b6"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.556348 20251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:50.570360 20251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:50.573766 20251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:50.573982 20251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:50.574599 20251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:50.575457 20251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:50.575525 20251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.575595 20251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:50.575635 20251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.585099 20251 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:33821
I20260807 18:38:50.585155 20455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:33821 every 8 connection(s)
I20260807 18:38:50.586081 20251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:50.589227 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 20251
I20260807 18:38:50.589380 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:38:50.589680 6697 external_mini_cluster.cc:1428] Setting key 60735cacfa29e681c3fa1a1e3a73bd25
Aug 07 18:38:50 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127930, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:38:50.599800 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:50.587939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:48363 (local address 127.6.138.126:44969)
0807 18:38:50.588103 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:50.588106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:50.589280 (+ 1174us) server_negotiation.cc:407] Connection header received
0807 18:38:50.590424 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:50.590428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:50.590485 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:50.590591 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:50.591893 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.592492 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.593643 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.593866 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.593975 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:50.596429 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:50.596432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:50.596434 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:50.596459 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:50.597736 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:50.598267 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:50.598270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:50.598272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:50.598360 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:50.598708 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:50.598711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:50.598713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:50.598903 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:50.599011 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:50.599510 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 18:38:50.599634 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:38:50.600744 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:50.588411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:48363)
0807 18:38:50.589116 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:50.589161 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:50.590194 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:50.590786 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:50.590796 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:50.591217 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:50.591723 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:50.591738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.592624 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.592627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:50.593505 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:50.593514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:50.594020 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:50.594024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:50.594212 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:50.594764 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:50.594780 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:50.596226 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:38:50.597870 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:50.597876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:50.597892 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:50.598161 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:50.598454 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:38:50.598457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:38:50.598460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:38:50.598611 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:38:50.599009 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:38:50.599017 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:38:50.599267 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 18:38:50.599461 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:38:50.599800 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":114,"threads_started":1}
I20260807 18:38:50.602042 20457 heartbeater.cc:347] Connected to a master server at 127.6.138.126:44969
I20260807 18:38:50.602329 20457 heartbeater.cc:464] Registering TS with master...
I20260807 18:38:50.602988 20457 heartbeater.cc:511] Master 127.6.138.126:44969 requested a full tablet report, sending...
I20260807 18:38:50.604034 19744 ts_manager.cc:194] Registered new tserver with Master: 8b8ceb9907324b9cb31e1e83243ddd10 (127.6.138.67:33821)
I20260807 18:38:50.604629 19744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:48363
I20260807 18:38:50.616014 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab.
Entry for principal oryx/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab.
WARNING: no policy specified for oryx/127.6.138.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab.
Entry for principal oryx/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab.
WARNING: no policy specified for oryx/127.6.138.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab.
Entry for principal oryx/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab.
I20260807 18:38:50.697209 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 19844
I20260807 18:38:50.711097 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 20044
I20260807 18:38:50.724388 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 20251
I20260807 18:38:50.734117 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 19685
I20260807 18:38:50.748622 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:44969
--webserver_interface=127.6.138.126
--webserver_port=36737
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:44969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:38:50.863867 20473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:50.864168 20473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:50.864210 20473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:50.868330 20473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:38:50.868392 20473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:50.868412 20473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:50.868430 20473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:38:50.868480 20473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:38:50.874347 20473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:44969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:44969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=36737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:50.875507 20473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:50.877103 20473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:50.886675 20479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:50.886686 20481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:50.886755 20473 server_base.cc:1034] running on GCE node
W20260807 18:38:50.886686 20483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:50.887611 20473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:50.888895 20473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:50.890062 20473 hybrid_clock.cc:648] HybridClock initialized: now 1786127930890047 us; error 33 us; skew 500 ppm
Aug 07 18:38:50 dist-test-slave-2d19 krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127930, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:50.892980 20473 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:38:50.893867 20473 webserver.cc:460] Webserver started at http://127.6.138.126:36737/ using document root <none> and password file <none>
I20260807 18:38:50.894407 20473 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:50.894479 20473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:50.898515 20473 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:38:50.900691 20495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:50.901929 20473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:38:50.902073 20473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "1154c74af65d41a695cb50cd093a6e14"
format_stamp: "Formatted at 2026-08-07 18:38:49 on dist-test-slave-2d19"
server_key: "00e1c8010a32c95448736d51228a5323"
server_key_iv: "b783782a3db2da5db6f087d11bf58303"
server_key_version: "encryptionkey@0"
I20260807 18:38:50.902647 20473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:50.925626 20473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:50.928716 20473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:50.928926 20473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:50.936503 20473 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:44969
I20260807 18:38:50.936527 20566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:44969 every 8 connection(s)
I20260807 18:38:50.937670 20473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:38:50.943387 20569 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:38:50.945343 20569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: Bootstrap starting.
I20260807 18:38:50.945946 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 20473
I20260807 18:38:50.946522 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:42429
--local_ip_for_outbound_sockets=127.6.138.65
--tserver_master_addrs=127.6.138.126:44969
--webserver_port=44433
--webserver_interface=127.6.138.65
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.138.65
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:38:50.965541 20569 log.cc:824] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: Log is configured to *not* fsync() on all Append() calls
I20260807 18:38:50.973006 20569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:38:50.973424 20569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: Bootstrap complete.
I20260807 18:38:50.978145 20569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } }
I20260807 18:38:50.978569 20569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1154c74af65d41a695cb50cd093a6e14, State: Initialized, Role: FOLLOWER
I20260807 18:38:50.979158 20569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } }
I20260807 18:38:50.979287 20569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:38:50.979379 20569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:38:50.979513 20569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:38:50.981726 20569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } }
I20260807 18:38:50.982098 20569 leader_election.cc:302] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1154c74af65d41a695cb50cd093a6e14; no voters:
I20260807 18:38:50.982473 20569 leader_election.cc:288] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:38:50.982666 20576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:38:50.982919 20576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [term 2 LEADER]: Becoming Leader. State: Replica: 1154c74af65d41a695cb50cd093a6e14, State: Running, Role: LEADER
I20260807 18:38:50.983256 20576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } }
I20260807 18:38:50.983862 20569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:38:50.984938 20582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1154c74af65d41a695cb50cd093a6e14. Latest consensus state: current_term: 2 leader_uuid: "1154c74af65d41a695cb50cd093a6e14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } } }
I20260807 18:38:50.985073 20582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:50.984944 20578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1154c74af65d41a695cb50cd093a6e14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1154c74af65d41a695cb50cd093a6e14" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 44969 } } }
I20260807 18:38:50.985469 20578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14 [sys.catalog]: This master's current role is: LEADER
I20260807 18:38:50.985795 20595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:38:50.989450 20595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:38:50.990239 20595 catalog_manager.cc:1259] Loaded cluster ID: 6f89b8fa503d4621a3dab22d25533700
I20260807 18:38:50.990317 20595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:38:50.992249 20595 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:38:50.992901 20595 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1154c74af65d41a695cb50cd093a6e14: Loaded TSK: 0
I20260807 18:38:50.993928 20595 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:38:51.074126 20572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:51.074402 20572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:51.074471 20572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:51.078495 20572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:51.078585 20572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:51.078678 20572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:38:51.084208 20572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.65.keytab
--principal=oryx/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:42429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=44433
--tserver_master_addrs=127.6.138.126:44969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:51.085429 20572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:51.086527 20572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:51.096599 20623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:51.096570 20627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:51.096581 20624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:51.096823 20572 server_base.cc:1034] running on GCE node
I20260807 18:38:51.097273 20572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:51.097944 20572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:51.099152 20572 hybrid_clock.cc:648] HybridClock initialized: now 1786127931099125 us; error 45 us; skew 500 ppm
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127931, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:51.102427 20572 init.cc:377] Logged in from keytab as oryx/127.6.138.65@KRBTEST.COM (short username oryx)
I20260807 18:38:51.103444 20572 webserver.cc:460] Webserver started at http://127.6.138.65:44433/ using document root <none> and password file <none>
I20260807 18:38:51.104058 20572 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:51.104171 20572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:51.108501 20572 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:38:51.110611 20634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:51.111889 20572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:51.112025 20572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "583c73637f0c4da0bc02a57f5f43eafd"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "20f340ba3275c5b470c9b3ea6fe688f1"
server_key_iv: "70795f71fe20a3d2e7e33e48e2ec12ea"
server_key_version: "encryptionkey@0"
I20260807 18:38:51.112627 20572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:51.137033 20572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:51.140501 20572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:51.140718 20572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:51.141372 20572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:51.142225 20572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:51.142295 20572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:51.142390 20572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:51.142421 20572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:51.151784 20572 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:42429
I20260807 18:38:51.151813 20831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:42429 every 8 connection(s)
I20260807 18:38:51.152923 20572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:38:51.154794 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 20572
I20260807 18:38:51.155314 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:39545
--local_ip_for_outbound_sockets=127.6.138.66
--tserver_master_addrs=127.6.138.126:44969
--webserver_port=37845
--webserver_interface=127.6.138.66
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.138.66
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:38:51.167260 20836 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:51.155199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:48077 (local address 127.6.138.126:44969)
0807 18:38:51.155911 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.155931 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.156515 (+ 584us) server_negotiation.cc:407] Connection header received
0807 18:38:51.157636 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.157646 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.157968 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.158315 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.159501 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.160604 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.161760 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.162243 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.162418 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.166541 (+ 4123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:48077: BlockingRecv error: recv got EOF from 127.6.138.65:48077 (error 108)
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":119,"threads_started":1}
I20260807 18:38:51.167418 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.155444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:48077)
0807 18:38:51.156316 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:51.156360 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:51.157202 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:51.158394 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:51.158401 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:51.158845 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:51.159315 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.159330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.160742 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.160745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.161628 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.161637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.162418 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.162422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.162608 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.163218 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:51.163235 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:51.166337 (+ 3102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":151,"threads_started":1}
W20260807 18:38:51.167994 20832 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:44969 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.138.126:44969: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:51.174023 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.168517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:37829)
0807 18:38:51.168770 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:51.168783 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:51.168852 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:51.169101 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:51.169103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:51.169236 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:51.169324 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.169329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.170367 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.170369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.170900 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.170908 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.171469 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.171471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.171539 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.172045 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:51.172056 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:51.173898 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184}
W20260807 18:38:51.174351 20836 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:51.168652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:37829 (local address 127.6.138.126:44969)
0807 18:38:51.168793 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.168797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.168815 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:38:51.168935 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.168938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.168991 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.169076 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.169425 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.170266 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.171014 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.171366 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.171467 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.174101 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:37829: BlockingRecv error: recv got EOF from 127.6.138.65:37829 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:51.179555 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.174612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:37021)
0807 18:38:51.174792 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:51.174806 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:51.174897 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:51.175143 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:51.175146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:51.175295 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:51.175361 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.175366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.176053 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.176058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.176619 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.176628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.177046 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.177050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.177126 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.177587 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:51.177598 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:51.179398 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116}
W20260807 18:38:51.179785 20832 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:38:51.179908 20836 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:51.174681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:37021 (local address 127.6.138.126:44969)
0807 18:38:51.174807 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.174812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.174851 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:38:51.175002 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.175009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.175045 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.175128 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.175474 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.175946 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.176735 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.176940 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.177044 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.179525 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:37021: BlockingRecv error: recv got EOF from 127.6.138.65:37021 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 18:38:51.275727 20838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:51.276075 20838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:51.276149 20838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:51.280247 20838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:51.280352 20838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:51.280442 20838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:38:51.286247 20838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.66.keytab
--principal=oryx/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:39545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=37845
--tserver_master_addrs=127.6.138.126:44969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:51.287473 20838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:51.288584 20838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:51.298512 20849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:51.298524 20848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:51.298524 20854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:51.298574 20838 server_base.cc:1034] running on GCE node
I20260807 18:38:51.299211 20838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:51.299854 20838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:51.301003 20838 hybrid_clock.cc:648] HybridClock initialized: now 1786127931300989 us; error 26 us; skew 500 ppm
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127931, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:51.303834 20838 init.cc:377] Logged in from keytab as oryx/127.6.138.66@KRBTEST.COM (short username oryx)
I20260807 18:38:51.304759 20838 webserver.cc:460] Webserver started at http://127.6.138.66:37845/ using document root <none> and password file <none>
I20260807 18:38:51.305326 20838 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:51.305424 20838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:51.309427 20838 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:38:51.311476 20872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:51.312765 20838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:38:51.312894 20838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "dcefddf26f384f8caaaea99abd6cf9d7"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "beb890cf615371e8b21441b376fe4c94"
server_key_iv: "0069c337cdf827853bd913d06884f47e"
server_key_version: "encryptionkey@0"
I20260807 18:38:51.313436 20838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:51.336167 20838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:51.339648 20838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:51.339856 20838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:51.340517 20838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:51.341378 20838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:51.341447 20838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:51.341529 20838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:51.341595 20838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:51.351258 20838 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:39545
I20260807 18:38:51.351285 21056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:39545 every 8 connection(s)
I20260807 18:38:51.352319 20838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:38:51.353039 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 20838
I20260807 18:38:51.353495 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:33821
--local_ip_for_outbound_sockets=127.6.138.67
--tserver_master_addrs=127.6.138.126:44969
--webserver_port=36517
--webserver_interface=127.6.138.67
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.138.67
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:38:51.364493 20836 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:51.354285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:45639 (local address 127.6.138.126:44969)
0807 18:38:51.354464 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.354470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.355662 (+ 1192us) server_negotiation.cc:407] Connection header received
0807 18:38:51.356666 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.356669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.356735 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.356811 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.357996 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.358859 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.360037 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.360495 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.360631 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.364382 (+ 3751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:45639: BlockingRecv error: recv got EOF from 127.6.138.66:45639 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:38:51.365461 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.354651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:45639)
0807 18:38:51.355477 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:51.355544 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:51.356441 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:51.356994 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:51.357001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:51.357348 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:51.357829 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.357842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.359019 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.359022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.359908 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.359919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.360652 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.360655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.360798 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.361443 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:51.361459 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:51.364211 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":120,"threads_started":1}
W20260807 18:38:51.365994 21057 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:44969 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.138.126:44969: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:51.371555 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.366489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:51451)
0807 18:38:51.366595 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:51.366611 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:51.366697 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:51.366938 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:51.366941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:51.367111 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:51.367205 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.367210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.367934 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.367938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.368551 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.368560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.368993 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.368997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.369099 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.369602 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:51.369614 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:51.371421 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 18:38:51.371683 20836 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:51.366599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:51451 (local address 127.6.138.126:44969)
0807 18:38:51.366727 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.366734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.366750 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:38:51.366792 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.366795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.366838 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.366921 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.367318 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.367827 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.368667 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.368885 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.368991 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.371552 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:51451: BlockingRecv error: recv got EOF from 127.6.138.66:51451 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:51.377424 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.372119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:53121)
0807 18:38:51.372292 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:51.372305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:51.372374 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:51.372602 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:51.372605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:51.372768 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:51.372857 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.372862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.373512 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.373517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.374449 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.374461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.374861 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.374865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.374948 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.375395 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:51.375409 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:51.377291 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260807 18:38:51.377585 20836 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:51.372208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:53121 (local address 127.6.138.126:44969)
0807 18:38:51.372329 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.372332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.372344 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:38:51.372466 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.372468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.372505 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.372579 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.372954 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.373418 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.374565 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.374763 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.374837 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.377422 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:53121: BlockingRecv error: recv got EOF from 127.6.138.66:53121 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 18:38:51.377749 21057 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:38:51.476189 21063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:38:51.476538 21063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:38:51.476584 21063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:38:51.480654 21063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:38:51.480726 21063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:38:51.480785 21063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:38:51.486649 21063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/krb5kdc/oryx_127.6.138.67.keytab
--principal=oryx/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:33821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=36517
--tserver_master_addrs=127.6.138.126:44969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:38:51.487721 21063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:38:51.488799 21063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:38:51.498677 21071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:51.498665 21072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:38:51.498654 21075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:38:51.499117 21063 server_base.cc:1034] running on GCE node
I20260807 18:38:51.499681 21063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:38:51.500442 21063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:38:51.501613 21063 hybrid_clock.cc:648] HybridClock initialized: now 1786127931501598 us; error 30 us; skew 500 ppm
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127931, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:38:51.504528 21063 init.cc:377] Logged in from keytab as oryx/127.6.138.67@KRBTEST.COM (short username oryx)
I20260807 18:38:51.505482 21063 webserver.cc:460] Webserver started at http://127.6.138.67:36517/ using document root <none> and password file <none>
I20260807 18:38:51.506110 21063 fs_manager.cc:359] Metadata directory not provided
I20260807 18:38:51.506204 21063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:38:51.510319 21063 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:38:51.512640 21083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:51.513957 21063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:38:51.514104 21063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "8b8ceb9907324b9cb31e1e83243ddd10"
format_stamp: "Formatted at 2026-08-07 18:38:50 on dist-test-slave-2d19"
server_key: "4a597686d003ccabe9d030341059970f"
server_key_iv: "37b33238d77952674939edc90bee35b6"
server_key_version: "encryptionkey@0"
I20260807 18:38:51.514678 21063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:38:51.544377 21063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:38:51.547501 21063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:38:51.547716 21063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:38:51.548377 21063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:38:51.549263 21063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:38:51.549311 21063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:51.549350 21063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:38:51.549403 21063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:38:51.558887 21063 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:33821
I20260807 18:38:51.558913 21257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:33821 every 8 connection(s)
I20260807 18:38:51.559929 21063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:38:51.563401 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 21063
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:38:51.572335 20836 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:51.562016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:39391 (local address 127.6.138.126:44969)
0807 18:38:51.562162 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.562166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.563280 (+ 1114us) server_negotiation.cc:407] Connection header received
0807 18:38:51.564467 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.564470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.564533 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.564626 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.565810 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.566671 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.568205 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.568512 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.568591 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.572178 (+ 3587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:39391: BlockingRecv error: recv got EOF from 127.6.138.67:39391 (error 108)
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":57}
I20260807 18:38:51.572983 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.562346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:39391)
0807 18:38:51.563082 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:51.563147 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:51.564120 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:51.564838 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:51.564845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:51.565216 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:51.565704 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.565718 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.566806 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.566809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.568095 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.568105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.568673 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.568679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.568831 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.569395 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:51.569420 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:51.572073 (+ 2653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":122,"threads_started":1}
W20260807 18:38:51.573446 21258 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:44969 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.138.126:44969: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
I20260807 18:38:51.574373 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.564519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46560 (local address 127.6.138.126:44969)
0807 18:38:51.564813 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.564816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.564826 (+ 10us) server_negotiation.cc:407] Connection header received
0807 18:38:51.564855 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.564857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.564887 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.564992 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.565752 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.566671 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.567913 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.568172 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.568301 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.570066 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:38:51.570071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:38:51.570085 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:38:51.570110 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:38:51.571889 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:51.572377 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:51.572397 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:51.572406 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:51.572639 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:38:51.572929 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:38:51.572932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:38:51.572934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:38:51.573416 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:38:51.573608 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:38:51.573928 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 18:38:51.574182 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":218,"thread_start_us":114,"threads_started":1}
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:51.579365 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.573908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:43439)
0807 18:38:51.574166 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:51.574177 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:51.574247 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:51.574492 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:51.574496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:51.574658 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:51.574761 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.574766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.575631 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.575634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.576430 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.576441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.577071 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.577074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.577191 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.577631 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:51.577642 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:51.579278 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177}
W20260807 18:38:51.579493 20836 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:51.573978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:43439 (local address 127.6.138.126:44969)
0807 18:38:51.574089 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.574093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.574233 (+ 140us) server_negotiation.cc:407] Connection header received
0807 18:38:51.574321 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.574324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.574365 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.574446 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.574862 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.575520 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.576685 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.576957 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.577068 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.579393 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:43439: BlockingRecv error: recv got EOF from 127.6.138.67:43439 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:51 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:51.584321 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:51.579940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:45431)
0807 18:38:51.580091 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:51.580110 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:51.580173 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:51.580425 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:51.580427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:51.580547 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:51.580616 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.580621 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.581258 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.581260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.581752 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:51.581758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.582134 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.582136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:51.582193 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.582566 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:51.582576 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:51.584185 (+ 1609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:38:51.584455 20836 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:51.580015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:45431 (local address 127.6.138.126:44969)
0807 18:38:51.580163 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:51.580166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:51.580178 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:38:51.580242 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:51.580243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:51.580333 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:51.580396 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:51.580707 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.581162 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.581845 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:51.582041 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:51.582111 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:51.584347 (+ 2236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:45431: BlockingRecv error: recv got EOF from 127.6.138.67:45431 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 18:38:51.584637 21258 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 18:38:52 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:52 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:52.186568 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:52.180520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:52433)
0807 18:38:52.180811 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:52.180830 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:52.180939 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:52.181187 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:52.181189 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:52.181377 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:52.181497 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:52.181502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.182427 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.182430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:52.183006 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:52.183014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.183509 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.183512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:52.183597 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:52.184199 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:52.184211 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:52.186421 (+ 2210us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
W20260807 18:38:52.186717 21267 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:52.180643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:52433 (local address 127.6.138.126:44969)
0807 18:38:52.180944 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:52.180948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:52.180992 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:38:52.181030 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:52.181032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:52.181076 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:52.181145 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:52.181641 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.182324 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.183141 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.183380 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.183486 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:52.186575 (+ 3089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:52433: BlockingRecv error: recv got EOF from 127.6.138.65:52433 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
Aug 07 18:38:52 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:52 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:52.384850 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:52.378791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:41549)
0807 18:38:52.379130 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:52.379153 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:52.379296 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:52.379611 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:52.379613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:52.379814 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:52.379923 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:52.379929 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.380717 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.380720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:52.381379 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:52.381408 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.381849 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.381851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:52.381923 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:52.382478 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:52.382491 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:52.384653 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":112,"threads_started":1}
W20260807 18:38:52.384881 21267 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:52.378870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:41549 (local address 127.6.138.126:44969)
0807 18:38:52.379022 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:52.379027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:52.379243 (+ 216us) server_negotiation.cc:407] Connection header received
0807 18:38:52.379413 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:52.379417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:52.379477 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:52.379556 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:52.380046 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.380602 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.381521 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.381745 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.381827 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:52.384776 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:41549: BlockingRecv error: recv got EOF from 127.6.138.66:41549 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 18:38:52 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:52 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:52.591543 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:52.585492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:56181)
0807 18:38:52.585829 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:52.585843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:52.585952 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:52.586268 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:52.586274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:52.586492 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:52.586595 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:52.586600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.587484 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.587487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:52.588276 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:52.588287 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.588732 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.588735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:52.588803 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:52.589396 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:52.589409 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:52.591377 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":117,"threads_started":1}
W20260807 18:38:52.591619 21267 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:52.585555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:56181 (local address 127.6.138.126:44969)
0807 18:38:52.585728 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:52.585733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:52.585917 (+ 184us) server_negotiation.cc:407] Connection header received
0807 18:38:52.586038 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:52.586040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:52.586092 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:52.586170 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:52.586732 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.587358 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.588367 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:52.588596 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:52.588681 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:52.591520 (+ 2839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:56181: BlockingRecv error: recv got EOF from 127.6.138.67:56181 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 18:38:53 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:53 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:53.193200 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:53.187457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:44233)
0807 18:38:53.187787 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:53.187805 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:53.187948 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:53.188312 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:53.188317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:53.188515 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:53.188634 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:53.188640 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.189430 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.189433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:53.190003 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:53.190011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.190465 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.190468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:53.190530 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:53.191102 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:53.191114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:53.193069 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":92,"threads_started":1}
W20260807 18:38:53.193372 21271 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:53.187651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:44233 (local address 127.6.138.126:44969)
0807 18:38:53.188064 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:53.188067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:53.188080 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:38:53.188115 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:53.188117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:53.188158 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:53.188237 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:53.188746 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.189326 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.190126 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.190359 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.190451 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:53.193221 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:44233: BlockingRecv error: recv got EOF from 127.6.138.65:44233 (error 108)
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":128,"threads_started":1}
Aug 07 18:38:53 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:53 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:53.391851 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:53.385779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:41441)
0807 18:38:53.386067 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:53.386084 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:53.386202 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:53.386453 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:53.386455 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:53.386660 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:53.386766 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:53.386772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.387886 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.387893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:53.388797 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:53.388807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.389234 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.389237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:53.389299 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:53.389805 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:53.389817 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:53.391706 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":92,"threads_started":1}
W20260807 18:38:53.392017 21271 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:53.385834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:41441 (local address 127.6.138.126:44969)
0807 18:38:53.386012 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:53.386019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:53.386164 (+ 145us) server_negotiation.cc:407] Connection header received
0807 18:38:53.386290 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:53.386292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:53.386338 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:53.386412 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:53.386901 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.387758 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.388913 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.389133 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.389217 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:53.391852 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:41441: BlockingRecv error: recv got EOF from 127.6.138.66:41441 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 18:38:53 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:53 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:53.597898 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:53.592391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:42773)
0807 18:38:53.592677 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:53.592692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:53.592791 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:53.593034 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:53.593036 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:53.593219 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:53.593321 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:53.593325 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.594076 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.594079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:53.594629 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:53.594637 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.595111 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.595114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:53.595173 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:53.595718 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:53.595729 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:53.597752 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":99,"threads_started":1}
W20260807 18:38:53.598073 21271 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:53.592547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:42773 (local address 127.6.138.126:44969)
0807 18:38:53.592682 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:53.592686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:53.592761 (+ 75us) server_negotiation.cc:407] Connection header received
0807 18:38:53.592873 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:53.592875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:53.592920 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:53.592993 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:53.593430 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.593971 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.594762 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:53.595002 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:53.595092 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:53.597899 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:42773: BlockingRecv error: recv got EOF from 127.6.138.67:42773 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 18:38:54 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:54 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:54.199856 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:54.194125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:34305)
0807 18:38:54.194456 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:54.194470 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:54.194577 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:54.194808 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:54.194809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:54.195031 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:54.195130 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:54.195135 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.195917 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.195920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:54.196613 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:54.196625 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.197121 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.197124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:54.197186 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:54.197720 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:54.197732 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:54.199728 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":117,"threads_started":1}
W20260807 18:38:54.200023 21277 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:54.194168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:34305 (local address 127.6.138.126:44969)
0807 18:38:54.194445 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:54.194449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:54.194540 (+ 91us) server_negotiation.cc:407] Connection header received
0807 18:38:54.194646 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:54.194648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:54.194692 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:54.194770 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:54.195257 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.195805 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.196738 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.197015 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.197100 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:54.199878 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:34305: BlockingRecv error: recv got EOF from 127.6.138.65:34305 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":85,"threads_started":1}
Aug 07 18:38:54 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:54 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:54.398901 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:54.392777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:51989)
0807 18:38:54.393092 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:54.393106 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:54.393240 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:54.393551 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:54.393553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:54.393796 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:54.393936 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:54.393943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.395052 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.395058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:54.395666 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:54.395677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.396161 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.396164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:54.396237 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:54.396883 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:54.396895 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:54.398779 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":101,"threads_started":1}
W20260807 18:38:54.399061 21277 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:54.392920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:51989 (local address 127.6.138.126:44969)
0807 18:38:54.393073 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:54.393077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:54.393183 (+ 106us) server_negotiation.cc:407] Connection header received
0807 18:38:54.393343 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:54.393345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:54.393392 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:54.393521 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:54.394104 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.394871 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.395805 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.396036 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.396127 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:54.398922 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:51989: BlockingRecv error: recv got EOF from 127.6.138.66:51989 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 18:38:54 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:54 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:54.604509 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:54.598782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:45901)
0807 18:38:54.599093 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:54.599108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:54.599246 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:54.599528 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:54.599531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:54.599755 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:54.599850 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:54.599856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.600721 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.600726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:54.601285 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:54.601293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.601758 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.601762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:54.601853 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:54.602390 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:54.602402 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:54.604357 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":89,"threads_started":1}
W20260807 18:38:54.604725 21277 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:54.598857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:45901 (local address 127.6.138.126:44969)
0807 18:38:54.599032 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:54.599037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:54.599185 (+ 148us) server_negotiation.cc:407] Connection header received
0807 18:38:54.599349 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:54.599354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:54.599409 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:54.599500 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:54.599976 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.600605 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.601411 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:54.601638 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:54.601758 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:54.604544 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:45901: BlockingRecv error: recv got EOF from 127.6.138.67:45901 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 18:38:55 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:55 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:55.212592 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:55.200787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:58429)
0807 18:38:55.201100 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:55.201114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:55.201216 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:55.201457 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:55.201459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:55.201643 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:55.201744 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:55.201749 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.202495 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.202498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:55.203091 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:55.203101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.203588 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.203592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:55.203673 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:55.210340 (+ 6667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:55.210359 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:55.212411 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":92,"threads_started":1}
W20260807 18:38:55.212824 21282 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:55.200925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:58429 (local address 127.6.138.126:44969)
0807 18:38:55.201185 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:55.201188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:55.201202 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:38:55.201291 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:55.201293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:55.201335 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:55.201414 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:55.201852 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.202384 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.203219 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.203453 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.203537 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:55.212560 (+ 9023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:58429: BlockingRecv error: recv got EOF from 127.6.138.65:58429 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
Aug 07 18:38:55 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:55 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:55.405097 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:55.399730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:58517)
0807 18:38:55.400010 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:55.400024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:55.400127 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:55.400380 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:55.400382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:55.400571 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:55.400670 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:55.400676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.401424 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.401427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:55.401981 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:55.401990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.402409 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.402412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:55.402472 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:55.402986 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:55.402997 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:55.404974 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":102,"threads_started":1}
W20260807 18:38:55.405251 21282 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:55.399788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:58517 (local address 127.6.138.126:44969)
0807 18:38:55.399948 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:55.399962 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:55.400097 (+ 135us) server_negotiation.cc:407] Connection header received
0807 18:38:55.400207 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:55.400210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:55.400275 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:55.400343 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:55.400804 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.401326 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.402089 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.402310 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.402388 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:55.405128 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:58517: BlockingRecv error: recv got EOF from 127.6.138.66:58517 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 18:38:55 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:55 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:55.611231 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:55.605359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:52865)
0807 18:38:55.605772 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:55.605786 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:55.605889 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:55.606188 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:55.606191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:55.606420 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:55.606550 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:55.606557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.607437 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.607440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:55.608016 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:55.608025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.608522 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.608525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:55.608591 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:55.609219 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:55.609232 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:55.611110 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":95,"threads_started":1}
W20260807 18:38:55.611423 21282 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:55.605476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:52865 (local address 127.6.138.126:44969)
0807 18:38:55.605627 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:55.605630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:55.605866 (+ 236us) server_negotiation.cc:407] Connection header received
0807 18:38:55.605958 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:55.605960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:55.606010 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:55.606082 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:55.606701 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.607303 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.608122 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:55.608402 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:55.608486 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:55.611253 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:52865: BlockingRecv error: recv got EOF from 127.6.138.67:52865 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:38:56 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:56 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:56.219913 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:56.213619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:46137)
0807 18:38:56.213938 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:56.214002 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:56.214133 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:56.214482 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:56.214485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:56.214736 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:56.214883 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:56.214893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.215804 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.215807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:56.216576 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:56.216587 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.217009 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.217012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:56.217116 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:56.217745 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:56.217759 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:56.219733 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":103,"threads_started":1}
W20260807 18:38:56.220011 21286 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:56.213730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:46137 (local address 127.6.138.126:44969)
0807 18:38:56.214006 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:56.214010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:56.214110 (+ 100us) server_negotiation.cc:407] Connection header received
0807 18:38:56.214238 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:56.214242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:56.214305 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:56.214426 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:56.215011 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.215695 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.216684 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.216906 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.216989 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:56.219870 (+ 2881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:46137: BlockingRecv error: recv got EOF from 127.6.138.65:46137 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":79,"threads_started":1}
Aug 07 18:38:56 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:56 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:56.412114 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:56.406096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:60709)
0807 18:38:56.406405 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:56.406419 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:56.406537 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:56.406820 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:56.406823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:56.407077 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:56.407179 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:56.407185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.408118 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.408123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:56.408748 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:56.408758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.409236 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.409239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:56.409329 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:56.409922 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:56.409935 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:56.411885 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":117,"threads_started":1}
W20260807 18:38:56.412145 21286 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:56.406227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:60709 (local address 127.6.138.126:44969)
0807 18:38:56.406375 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:56.406378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:56.406494 (+ 116us) server_negotiation.cc:407] Connection header received
0807 18:38:56.406622 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:56.406624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:56.406693 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:56.406777 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:56.407328 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.407994 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.408883 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.409114 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.409229 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:56.412042 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:60709: BlockingRecv error: recv got EOF from 127.6.138.66:60709 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 18:38:56 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:56 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:56.618700 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:56.612193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:36837)
0807 18:38:56.612547 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:56.612567 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:56.612737 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:56.613083 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:56.613087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:56.613326 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:56.613427 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:56.613433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.614309 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.614315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:56.614960 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:56.614969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.615516 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.615520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:56.615623 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:56.616248 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:56.616328 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:56.618505 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":138,"threads_started":1}
W20260807 18:38:56.618779 21286 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:56.612358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:36837 (local address 127.6.138.126:44969)
0807 18:38:56.612547 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:56.612553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:56.612671 (+ 118us) server_negotiation.cc:407] Connection header received
0807 18:38:56.612868 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:56.612872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:56.612943 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:56.613048 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:56.613565 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.614179 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.615101 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:56.615371 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:56.615494 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:56.618655 (+ 3161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:36837: BlockingRecv error: recv got EOF from 127.6.138.67:36837 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 18:38:57 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:57 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:57.226609 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:57.220732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:50719)
0807 18:38:57.221020 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:57.221037 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:57.221151 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:57.221439 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:57.221442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:57.221660 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:57.221755 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:57.221761 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.222552 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.222556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:57.223119 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:57.223128 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.223618 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.223622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:57.223711 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:57.224340 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:57.224353 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:57.226485 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
W20260807 18:38:57.226845 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:57.220812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:50719 (local address 127.6.138.126:44969)
0807 18:38:57.221133 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:57.221137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:57.221152 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:38:57.221258 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:57.221261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:57.221314 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:57.221410 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:57.221881 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.222427 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.223245 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.223488 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.223601 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:57.226680 (+ 3079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:50719: BlockingRecv error: recv got EOF from 127.6.138.65:50719 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":79,"threads_started":1}
Aug 07 18:38:57 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:57 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:57.419085 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:57.413153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:55515)
0807 18:38:57.413466 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:57.413480 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:57.413580 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:57.413810 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:57.413812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:57.414001 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:57.414098 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:57.414103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.414841 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.414844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:57.415502 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:57.415512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.416004 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.416008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:57.416092 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:57.416748 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:57.416761 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:57.418938 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":116,"threads_started":1}
W20260807 18:38:57.419225 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:57.413204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:55515 (local address 127.6.138.126:44969)
0807 18:38:57.413364 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:57.413368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:57.413548 (+ 180us) server_negotiation.cc:407] Connection header received
0807 18:38:57.413650 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:57.413652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:57.413696 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:57.413769 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:57.414196 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.414710 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.415623 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.415846 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.415919 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:57.419099 (+ 3180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:55515: BlockingRecv error: recv got EOF from 127.6.138.66:55515 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 18:38:57 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:57 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:57.625799 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:57.619625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:43865)
0807 18:38:57.619946 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:57.619974 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:57.620090 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:57.620430 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:57.620432 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:57.620729 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:57.620838 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:57.620845 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.621678 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.621683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:57.622238 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:57.622246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.622723 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.622727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:57.622814 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:57.623397 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:57.623410 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:57.625619 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":145,"threads_started":1}
W20260807 18:38:57.625860 21296 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:57.619799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:43865 (local address 127.6.138.126:44969)
0807 18:38:57.619945 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:57.619949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:57.620068 (+ 119us) server_negotiation.cc:407] Connection header received
0807 18:38:57.620206 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:57.620209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:57.620293 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:57.620395 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:57.620979 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.621557 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.622366 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:57.622602 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:57.622717 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:57.625752 (+ 3035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:43865: BlockingRecv error: recv got EOF from 127.6.138.67:43865 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 18:38:58 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:58 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:58.233580 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:58.227524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:59651)
0807 18:38:58.228137 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:58.228156 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:58.228328 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:58.228647 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:58.228650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:58.228874 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:58.228969 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:58.228975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.229827 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.229831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:58.230461 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:58.230471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.230904 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.230906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:58.231001 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:58.231507 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:58.231518 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:58.233459 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":123,"threads_started":1}
W20260807 18:38:58.233784 21300 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:58.227625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:59651 (local address 127.6.138.126:44969)
0807 18:38:58.227915 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:58.227921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:58.228273 (+ 352us) server_negotiation.cc:407] Connection header received
0807 18:38:58.228445 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:58.228448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:58.228504 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:58.228609 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:58.229108 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.229667 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.230580 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.230802 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.230881 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:58.233613 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:59651: BlockingRecv error: recv got EOF from 127.6.138.65:59651 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":101,"threads_started":1}
Aug 07 18:38:58 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:58 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:58.425981 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:58.419946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:42103)
0807 18:38:58.420373 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:58.420388 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:58.420494 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:58.420738 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:58.420741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:58.420933 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:58.421030 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:58.421035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.422031 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.422034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:58.422648 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:58.422658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.423153 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.423156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:58.423223 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:58.423832 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:58.423849 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:58.425835 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":143,"threads_started":1}
W20260807 18:38:58.426126 21300 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:58.419991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:42103 (local address 127.6.138.126:44969)
0807 18:38:58.420176 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:58.420181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:58.420484 (+ 303us) server_negotiation.cc:407] Connection header received
0807 18:38:58.420568 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:58.420570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:58.420621 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:58.420704 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:58.421171 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.421769 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.422777 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.423021 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.423113 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:58.425996 (+ 2883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:42103: BlockingRecv error: recv got EOF from 127.6.138.66:42103 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:38:58 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:58 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:58.632905 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:58.626898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:51091)
0807 18:38:58.627323 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:58.627337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:58.627478 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:58.627780 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:58.627784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:58.628004 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:58.628113 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:58.628120 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.628953 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.628958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:58.629520 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:58.629528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.630011 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.630015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:58.630108 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:58.630693 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:58.630706 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:58.632705 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":97,"threads_started":1}
W20260807 18:38:58.632949 21300 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:58.627085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:51091 (local address 127.6.138.126:44969)
0807 18:38:58.627255 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:58.627262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:58.627417 (+ 155us) server_negotiation.cc:407] Connection header received
0807 18:38:58.627593 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:58.627598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:58.627655 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:58.627747 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:58.628278 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.628830 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.629660 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:58.629887 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:58.630002 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:58.632840 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:51091: BlockingRecv error: recv got EOF from 127.6.138.67:51091 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 18:38:59 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:59 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:59.241663 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:59.234349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:54141)
0807 18:38:59.234660 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:59.234674 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:59.234769 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:59.235120 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:59.235125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:59.235365 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:59.235506 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:59.235516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.236468 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.236472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:59.237441 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:59.237454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.238056 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.238061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:59.238154 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:59.238929 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:59.238948 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:59.241489 (+ 2541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":114,"threads_started":1}
W20260807 18:38:59.241778 21308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:59.234494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:54141 (local address 127.6.138.126:44969)
0807 18:38:59.234745 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:59.234751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:59.234773 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:38:59.234872 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:59.234876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:59.234941 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:59.235062 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:59.235631 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.236355 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.237603 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.237919 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.238015 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:59.241642 (+ 3627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:54141: BlockingRecv error: recv got EOF from 127.6.138.65:54141 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":89,"threads_started":1}
Aug 07 18:38:59 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:59 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:59.433429 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:59.427062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:55459)
0807 18:38:59.427396 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:59.427410 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:59.427523 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:59.427865 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:59.427868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:59.428097 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:59.428202 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:59.428209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.429455 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.429458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:59.430083 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:59.430094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.430591 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.430594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:59.430666 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:59.431255 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:59.431268 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:59.433249 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":124,"threads_started":1}
W20260807 18:38:59.433471 21308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:59.427167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:55459 (local address 127.6.138.126:44969)
0807 18:38:59.427340 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:59.427347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:59.427520 (+ 173us) server_negotiation.cc:407] Connection header received
0807 18:38:59.427653 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:59.427656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:59.427715 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:59.427819 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:59.428370 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.429304 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.430215 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.430443 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.430543 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:59.433373 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:55459: BlockingRecv error: recv got EOF from 127.6.138.66:55459 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 18:38:59 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:38:59 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:38:59.639499 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:38:59.633876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:51793)
0807 18:38:59.634200 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 18:38:59.634217 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:38:59.634359 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:38:59.634641 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:38:59.634644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:38:59.634854 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:38:59.634958 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:59.634964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.635727 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.635732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:59.636349 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:38:59.636358 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.636820 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.636823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:38:59.636910 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:59.637472 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:38:59.637485 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:38:59.639374 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":95,"threads_started":1}
W20260807 18:38:59.639700 21308 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:38:59.634022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:51793 (local address 127.6.138.126:44969)
0807 18:38:59.634194 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:38:59.634198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:38:59.634294 (+ 96us) server_negotiation.cc:407] Connection header received
0807 18:38:59.634467 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:38:59.634470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:38:59.634523 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:38:59.634611 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:38:59.635081 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.635613 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.636465 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:38:59.636695 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:38:59.636804 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:38:59.639519 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:51793: BlockingRecv error: recv got EOF from 127.6.138.67:51793 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 18:39:00 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:00 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:00.251624 21311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:00.242453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:52061)
0807 18:39:00.242748 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:00.242770 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:00.242865 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:00.243133 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:00.243136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:00.243333 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:00.243442 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:00.243447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.244332 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.244335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:00.244955 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:00.244967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.245425 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.245427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:00.245518 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:00.249210 (+ 3692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:00.249223 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:00.251419 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":89,"threads_started":1}
W20260807 18:39:00.251677 21312 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:00.242547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:52061 (local address 127.6.138.126:44969)
0807 18:39:00.242812 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:00.242815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:00.242830 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:39:00.242938 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:00.242940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:00.242982 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:00.243062 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:00.243552 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.244154 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.245081 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.245319 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.245402 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:00.251552 (+ 6150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:52061: BlockingRecv error: recv got EOF from 127.6.138.65:52061 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
Aug 07 18:39:00 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:00 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:00.440933 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:00.434365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:38703)
0807 18:39:00.434685 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:00.434697 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:00.434842 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:00.435147 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:00.435150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:00.435374 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:00.435475 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:00.435481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.436342 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.436345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:00.436908 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:00.436917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.437394 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.437398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:00.437489 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:00.438131 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:00.438149 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:00.440780 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":98,"threads_started":1}
W20260807 18:39:00.441080 21312 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:00.434430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:38703 (local address 127.6.138.126:44969)
0807 18:39:00.434616 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:00.434620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:00.434779 (+ 159us) server_negotiation.cc:407] Connection header received
0807 18:39:00.434949 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:00.434952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:00.435005 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:00.435121 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:00.435627 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.436186 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.437037 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.437273 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.437388 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:00.440954 (+ 3566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:38703: BlockingRecv error: recv got EOF from 127.6.138.66:38703 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 18:39:00 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:00 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:00.646121 21315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:00.640402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:60505)
0807 18:39:00.640707 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:00.640720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:00.640850 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:00.641150 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:00.641153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:00.641400 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:00.641501 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:00.641507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.642274 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.642279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:00.642860 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:00.642868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.643350 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.643354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:00.643441 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:00.643940 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:00.643989 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:00.645983 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":98,"threads_started":1}
W20260807 18:39:00.646299 21312 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:00.640415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:60505 (local address 127.6.138.126:44969)
0807 18:39:00.640587 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:00.640594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:00.640796 (+ 202us) server_negotiation.cc:407] Connection header received
0807 18:39:00.640960 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:00.640963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:00.641021 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:00.641121 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:00.641629 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.642162 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.643004 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:00.643233 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:00.643345 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:00.646133 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:60505: BlockingRecv error: recv got EOF from 127.6.138.67:60505 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 18:39:01 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:01 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:01.258730 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:01.252585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:60723)
0807 18:39:01.252904 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:01.252917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:01.253041 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:01.253392 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:01.253395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:01.253622 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:01.253726 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:01.253732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.254734 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.254736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:01.255303 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:01.255311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.255839 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.255842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:01.255914 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:01.256532 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:01.256562 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:01.258538 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":82,"threads_started":1}
W20260807 18:39:01.258797 21317 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:01.252671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:60723 (local address 127.6.138.126:44969)
0807 18:39:01.252939 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:01.252946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:01.252968 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:39:01.253147 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:01.253149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:01.253223 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:01.253327 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:01.253874 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.254604 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.255412 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.255720 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.255823 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:01.258667 (+ 2844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:60723: BlockingRecv error: recv got EOF from 127.6.138.65:60723 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":96,"threads_started":1}
Aug 07 18:39:01 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:01 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:01.447772 21318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:01.441966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:54725)
0807 18:39:01.442322 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:01.442340 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:01.442480 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:01.442775 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:01.442778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:01.443039 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:01.443136 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:01.443142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.443936 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.443941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:01.444594 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:01.444604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.445070 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.445074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:01.445163 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:01.445714 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:01.445725 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:01.447587 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":100,"threads_started":1}
W20260807 18:39:01.447809 21317 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:01.442125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:54725 (local address 127.6.138.126:44969)
0807 18:39:01.442291 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:01.442296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:01.442420 (+ 124us) server_negotiation.cc:407] Connection header received
0807 18:39:01.442605 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:01.442608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:01.442664 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:01.442751 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:01.443257 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.443822 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.444716 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.444957 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.445066 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:01.447709 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:54725: BlockingRecv error: recv got EOF from 127.6.138.66:54725 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 18:39:01 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:01 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:01.653008 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:01.646989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:39379)
0807 18:39:01.647465 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:01.647482 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:01.647622 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:01.647916 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:01.647919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:01.648142 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:01.648237 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:01.648242 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.649069 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.649072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:01.649724 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:01.649734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.650236 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.650243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:01.650352 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:01.650940 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:01.650952 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:01.652889 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":137,"threads_started":1}
W20260807 18:39:01.653168 21317 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:01.647184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:39379 (local address 127.6.138.126:44969)
0807 18:39:01.647490 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:01.647497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:01.647568 (+ 71us) server_negotiation.cc:407] Connection header received
0807 18:39:01.647732 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:01.647736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:01.647792 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:01.647882 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:01.648396 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.648926 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.649846 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:01.650098 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:01.650200 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:01.653037 (+ 2837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:39379: BlockingRecv error: recv got EOF from 127.6.138.67:39379 (error 108)
Metrics: {"server-negotiator.queue_time_us":170}
Aug 07 18:39:02 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:02 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:02.265566 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:02.259717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:48155)
0807 18:39:02.260077 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:02.260091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:02.260201 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:02.260505 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:02.260507 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:02.260691 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:02.260787 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:02.260792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.261669 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.261673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:02.262419 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:02.262450 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.262918 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.262921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:02.262986 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:02.263505 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:02.263516 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:02.265448 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":95,"threads_started":1}
W20260807 18:39:02.265766 21321 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:02.259862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:48155 (local address 127.6.138.126:44969)
0807 18:39:02.260108 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:02.260112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:02.260176 (+ 64us) server_negotiation.cc:407] Connection header received
0807 18:39:02.260332 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:02.260334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:02.260381 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:02.260472 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:02.260891 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.261528 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.262561 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.262811 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.262895 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:02.265614 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:48155: BlockingRecv error: recv got EOF from 127.6.138.65:48155 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":63,"threads_started":1}
Aug 07 18:39:02 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:02 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:02.455304 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:02.448717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:47191)
0807 18:39:02.449225 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:02.449244 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:02.449430 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:02.449748 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:02.449752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:02.449996 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:02.450099 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:02.450105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.450955 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.450960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:02.451582 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:02.451591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.452116 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.452120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:02.452219 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:02.452824 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:02.452837 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:02.455096 (+ 2259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":117,"threads_started":1}
W20260807 18:39:02.455368 21321 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:02.448880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:47191 (local address 127.6.138.126:44969)
0807 18:39:02.449226 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:02.449233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:02.449346 (+ 113us) server_negotiation.cc:407] Connection header received
0807 18:39:02.449554 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:02.449559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:02.449619 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:02.449722 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:02.450236 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.450821 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.451718 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.451983 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.452110 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:02.455249 (+ 3139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:47191: BlockingRecv error: recv got EOF from 127.6.138.66:47191 (error 108)
Metrics: {"server-negotiator.queue_time_us":239}
Aug 07 18:39:02 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:02 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:02.659924 21328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:02.653892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:43935)
0807 18:39:02.654220 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:02.654242 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:02.654347 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:02.654589 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:02.654592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:02.654784 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:02.654885 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:02.654890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.655805 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.655807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:02.656412 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:02.656420 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.656897 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.656900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:02.656960 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:02.657547 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:02.657562 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:02.659719 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":115,"threads_started":1}
W20260807 18:39:02.659986 21321 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:02.653987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:43935 (local address 127.6.138.126:44969)
0807 18:39:02.654167 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:02.654170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:02.654318 (+ 148us) server_negotiation.cc:407] Connection header received
0807 18:39:02.654416 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:02.654418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:02.654464 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:02.654542 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:02.655044 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.655690 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.656536 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:02.656786 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:02.656881 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:02.659863 (+ 2982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:43935: BlockingRecv error: recv got EOF from 127.6.138.67:43935 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 18:39:03 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:03 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:03.272159 21329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:03.266414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:60825)
0807 18:39:03.266746 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:03.266760 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:03.266855 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:03.267108 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:03.267110 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:03.267306 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:03.267406 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:03.267411 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.268192 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.268194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:03.268935 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:03.268947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.269388 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.269391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:03.269454 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:03.269948 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:03.269960 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:03.271941 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":102,"threads_started":1}
W20260807 18:39:03.272217 21330 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:03.266580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:60825 (local address 127.6.138.126:44969)
0807 18:39:03.266810 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:03.266814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:03.266827 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:39:03.266941 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:03.266943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:03.266993 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:03.267069 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:03.267531 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.268087 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.269053 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.269283 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.269368 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:03.272089 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:60825: BlockingRecv error: recv got EOF from 127.6.138.65:60825 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":64,"threads_started":1}
Aug 07 18:39:03 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:03 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:03.462467 21333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:03.456236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:39691)
0807 18:39:03.456612 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:03.456630 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:03.456758 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:03.457044 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:03.457047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:03.457269 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:03.457364 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:03.457370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.458186 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.458192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:03.458762 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:03.458770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.459261 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.459265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:03.459358 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:03.460121 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:03.460134 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:03.462330 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":136,"threads_started":1}
W20260807 18:39:03.462653 21330 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:03.456369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:39691 (local address 127.6.138.126:44969)
0807 18:39:03.456582 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:03.456587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:03.456717 (+ 130us) server_negotiation.cc:407] Connection header received
0807 18:39:03.456848 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:03.456851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:03.456909 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:03.457012 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:03.457519 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.458067 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.458897 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.459142 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.459254 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:03.462507 (+ 3253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:39691: BlockingRecv error: recv got EOF from 127.6.138.66:39691 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 18:39:03 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:03 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:03.667011 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:03.661004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:46993)
0807 18:39:03.661410 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:03.661427 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:03.661578 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:03.661888 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:03.661891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:03.662118 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:03.662220 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:03.662227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.663010 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.663015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:03.663590 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:03.663599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.664108 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.664111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:03.664198 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:03.664881 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:03.664894 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:03.666876 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":113,"threads_started":1}
W20260807 18:39:03.667212 21330 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:03.661195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:46993 (local address 127.6.138.126:44969)
0807 18:39:03.661379 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:03.661383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:03.661522 (+ 139us) server_negotiation.cc:407] Connection header received
0807 18:39:03.661686 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:03.661689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:03.661747 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:03.661848 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:03.662357 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.662890 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.663723 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:03.663962 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:03.664100 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:03.667072 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:46993: BlockingRecv error: recv got EOF from 127.6.138.67:46993 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 18:39:04 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:04 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:04.279014 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:04.273062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:55235)
0807 18:39:04.273407 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:04.273424 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:04.273542 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:04.273828 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:04.273831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:04.274035 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:04.274161 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:04.274167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.274930 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.274933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:04.275521 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:04.275530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.275981 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.275984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:04.276073 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:04.276646 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:04.276658 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:04.278822 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":93,"threads_started":1}
W20260807 18:39:04.279143 21336 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:04.273132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:55235 (local address 127.6.138.126:44969)
0807 18:39:04.273407 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:04.273411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:04.273504 (+ 93us) server_negotiation.cc:407] Connection header received
0807 18:39:04.273628 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:04.273630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:04.273675 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:04.273756 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:04.274271 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.274823 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.275646 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.275876 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.275968 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:04.278984 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:55235: BlockingRecv error: recv got EOF from 127.6.138.65:55235 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
Aug 07 18:39:04 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:04 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:04.468906 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:04.463360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:47535)
0807 18:39:04.463656 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:04.463670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:04.463761 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:04.464030 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:04.464033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:04.464223 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:04.464355 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:04.464361 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.465120 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.465123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:04.465731 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:04.465740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.466202 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.466204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:04.466272 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:04.466787 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:04.466798 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:04.468769 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":117,"threads_started":1}
W20260807 18:39:04.469071 21336 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:04.463527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:47535 (local address 127.6.138.126:44969)
0807 18:39:04.463692 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:04.463696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:04.463713 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:39:04.463848 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:04.463850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:04.463893 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:04.463968 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:04.464458 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.464987 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.465845 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.466093 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.466173 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:04.468918 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:47535: BlockingRecv error: recv got EOF from 127.6.138.66:47535 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 18:39:04 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:04 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:04.673966 21338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:04.667817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:45565)
0807 18:39:04.668286 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:04.668298 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:04.668432 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:04.668720 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:04.668724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:04.668942 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:04.669036 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:04.669042 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.669812 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.669817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:04.670401 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:04.670410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.670873 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.670876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:04.670973 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:04.671486 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:04.671498 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:04.673756 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":155,"threads_started":1}
W20260807 18:39:04.674027 21336 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:04.667937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:45565 (local address 127.6.138.126:44969)
0807 18:39:04.668113 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:04.668121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:04.668380 (+ 259us) server_negotiation.cc:407] Connection header received
0807 18:39:04.668547 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:04.668550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:04.668604 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:04.668693 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:04.669161 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.669697 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.670525 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:04.670757 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:04.670868 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:04.673900 (+ 3032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:45565: BlockingRecv error: recv got EOF from 127.6.138.67:45565 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 18:39:05 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:05 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:05.290303 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:05.279871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:40129)
0807 18:39:05.280286 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:05.280304 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:05.280412 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:05.280668 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:05.280670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:05.280849 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:05.280945 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:05.280950 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.281698 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.281700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:05.282270 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:05.282279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.282770 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.282774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:05.282857 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:05.288033 (+ 5176us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:05.288047 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:05.290160 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":100,"threads_started":1}
W20260807 18:39:05.290535 21340 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:05.279932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:40129 (local address 127.6.138.126:44969)
0807 18:39:05.280336 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:05.280340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:05.280358 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:39:05.280504 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:05.280506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:05.280551 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:05.280632 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:05.281057 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.281600 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.282379 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.282631 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.282734 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:05.290369 (+ 7635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:40129: BlockingRecv error: recv got EOF from 127.6.138.65:40129 (error 108)
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":63,"threads_started":1}
Aug 07 18:39:05 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:05 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:05.476060 21341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:05.469955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:56029)
0807 18:39:05.470311 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:05.470324 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:05.470433 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:05.470731 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:05.470734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:05.470927 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:05.471045 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:05.471051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.471822 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.471825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:05.472473 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:05.472484 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.473017 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.473022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:05.473115 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:05.473725 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:05.473741 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:05.475879 (+ 2138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":81,"threads_started":1}
W20260807 18:39:05.476162 21340 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:05.470137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:56029 (local address 127.6.138.126:44969)
0807 18:39:05.470296 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:05.470300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:05.470396 (+ 96us) server_negotiation.cc:407] Connection header received
0807 18:39:05.470548 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:05.470550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:05.470599 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:05.470681 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:05.471161 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.471688 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.472598 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.472876 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.472969 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:05.476034 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:56029: BlockingRecv error: recv got EOF from 127.6.138.66:56029 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:39:05 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:05 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.67@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:05.680925 21344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:05.674806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.67:51603)
0807 18:39:05.675156 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:05.675170 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:05.675336 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:05.675678 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:05.675680 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:05.675946 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:05.676102 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:05.676109 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.677155 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.677158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:05.677734 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:05.677742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.678146 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.678149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:05.678210 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:05.678703 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:05.678715 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:05.680747 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":96,"threads_started":1}
W20260807 18:39:05.680991 21340 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:05.674908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:51603 (local address 127.6.138.126:44969)
0807 18:39:05.675049 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:05.675053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:05.675278 (+ 225us) server_negotiation.cc:407] Connection header received
0807 18:39:05.675447 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:05.675451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:05.675514 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:05.675617 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:05.676279 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.677046 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.677837 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:05.678050 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:05.678123 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:05.680881 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.67:51603: BlockingRecv error: recv got EOF from 127.6.138.67:51603 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 18:39:06 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:06 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.65@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:06.299081 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:06.291480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.65:40663)
0807 18:39:06.291841 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:06.291856 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:06.292012 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:06.292371 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:06.292377 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:06.292684 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:06.292859 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:06.292871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.293869 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.293874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:06.295141 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:06.295155 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.295680 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.295685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:06.295822 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:06.296537 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:06.296562 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:06.298936 (+ 2374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":100,"threads_started":1}
W20260807 18:39:06.299378 21347 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:06.291525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:40663 (local address 127.6.138.126:44969)
0807 18:39:06.291801 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:06.291806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:06.291994 (+ 188us) server_negotiation.cc:407] Connection header received
0807 18:39:06.292118 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:06.292121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:06.292174 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:06.292299 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:06.293025 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.293751 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.295299 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.295540 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.295637 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:06.299158 (+ 3521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.65:40663: BlockingRecv error: recv got EOF from 127.6.138.65:40663 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":67,"threads_started":1}
Aug 07 18:39:06 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:39:06 dist-test-slave-2d19 krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.138.66@KRBTEST.COM for oryx/127.6.138.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:39:06.482725 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:06.477139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:44969 (local address 127.6.138.66:41335)
0807 18:39:06.477418 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:06.477432 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:06.477532 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:06.477786 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:06.477788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:06.477992 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:06.478089 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:06.478094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.478800 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.478803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:06.479399 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:06.479408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.479840 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.479843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:06.479970 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:06.480648 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:06.480660 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:06.482591 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.138.126:44969: Server oryx/127.6.138.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
W20260807 18:39:06.482908 21347 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:06.477255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:41335 (local address 127.6.138.126:44969)
0807 18:39:06.477450 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:06.477453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:06.477485 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:39:06.477625 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:06.477627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:06.477672 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:06.477746 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:06.478193 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.478708 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.479502 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.479712 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.479787 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:06.482740 (+ 2953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.138.66:41335: BlockingRecv error: recv got EOF from 127.6.138.66:41335 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:39:06.565274 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 20572
I20260807 18:39:06.580305 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 20838
I20260807 18:39:06.591943 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 21063
I20260807 18:39:06.604990 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 20473
2026-08-07T18:39:06Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18477 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T18:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:39:06Z Disabled control of system clock
I20260807 18:39:06.643100 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:33409
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:33409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:39:06.756410 21355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:06.756685 21355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:06.756719 21355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:06.760731 21355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:39:06.760783 21355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:06.760798 21355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:06.760815 21355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:39:06.760830 21355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:39:06.765408 21355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:33409
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:33409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:06.766453 21355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:06.767786 21355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:06.776880 21363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:06.776863 21365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:06.776868 21367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:06.776909 21355 server_base.cc:1034] running on GCE node
I20260807 18:39:06.777700 21355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:06.778858 21355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:06.780021 21355 hybrid_clock.cc:648] HybridClock initialized: now 1786127946779989 us; error 46 us; skew 500 ppm
I20260807 18:39:06.782004 21355 webserver.cc:460] Webserver started at http://127.6.138.126:46341/ using document root <none> and password file <none>
I20260807 18:39:06.782622 21355 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:06.782689 21355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:06.782915 21355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:06.785069 21355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "15826e199fd740ab8ab6e838b509332d"
format_stamp: "Formatted at 2026-08-07 18:39:06 on dist-test-slave-2d19"
server_key: "65c50d5e9978827eb1acbafd0ef72c51"
server_key_iv: "8b01fe6d0bacfff4fa0e065324145c2d"
server_key_version: "encryptionkey@0"
I20260807 18:39:06.785655 21355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "15826e199fd740ab8ab6e838b509332d"
format_stamp: "Formatted at 2026-08-07 18:39:06 on dist-test-slave-2d19"
server_key: "65c50d5e9978827eb1acbafd0ef72c51"
server_key_iv: "8b01fe6d0bacfff4fa0e065324145c2d"
server_key_version: "encryptionkey@0"
I20260807 18:39:06.789445 21355 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 18:39:06.791879 21375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:06.793211 21355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:39:06.793342 21355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "15826e199fd740ab8ab6e838b509332d"
format_stamp: "Formatted at 2026-08-07 18:39:06 on dist-test-slave-2d19"
server_key: "65c50d5e9978827eb1acbafd0ef72c51"
server_key_iv: "8b01fe6d0bacfff4fa0e065324145c2d"
server_key_version: "encryptionkey@0"
I20260807 18:39:06.793457 21355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:06.810230 21355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:06.811021 21355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:06.811244 21355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:06.819006 21355 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:33409
I20260807 18:39:06.819104 21464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:33409 every 8 connection(s)
I20260807 18:39:06.820017 21355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:39:06.823388 21465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:39:06.828958 21465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d: Bootstrap starting.
I20260807 18:39:06.829582 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 21355
I20260807 18:39:06.829689 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:39:06.829988 6697 external_mini_cluster.cc:1428] Setting key 4fef2774b352a8549b8690d724dd067b
I20260807 18:39:06.831740 21465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d: Neither blocks nor log segments found. Creating new log.
I20260807 18:39:06.832619 21465 log.cc:824] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:39:06.834563 21465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d: No bootstrap required, opened a new log
I20260807 18:39:06.838963 21465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15826e199fd740ab8ab6e838b509332d" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33409 } }
I20260807 18:39:06.839254 21465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:39:06.839341 21465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15826e199fd740ab8ab6e838b509332d, State: Initialized, Role: FOLLOWER
I20260807 18:39:06.839381 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:06.831318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.6.138.126:33409)
0807 18:39:06.832019 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:06.832035 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:06.832075 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:39:06.832763 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:06.832789 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:06.833203 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:06.833534 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:06.833968 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.834988 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.835926 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:06.836475 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:06.836657 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:06.837536 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:06.837542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:06.837559 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:39:06.837947 (+ 388us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:06.838005 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:39:06.838026 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:06.838146 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:06.838314 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 18:39:06.838578 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":146,"threads_started":1}
I20260807 18:39:06.839928 21465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15826e199fd740ab8ab6e838b509332d" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33409 } }
I20260807 18:39:06.840103 21465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:39:06.840170 21465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:39:06.840292 21465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:39:06.841305 21465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15826e199fd740ab8ab6e838b509332d" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33409 } }
I20260807 18:39:06.841703 21465 leader_election.cc:302] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15826e199fd740ab8ab6e838b509332d; no voters:
I20260807 18:39:06.842078 21465 leader_election.cc:288] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:39:06.842226 21473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:39:06.842473 21473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [term 1 LEADER]: Becoming Leader. State: Replica: 15826e199fd740ab8ab6e838b509332d, State: Running, Role: LEADER
I20260807 18:39:06.842849 21473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15826e199fd740ab8ab6e838b509332d" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33409 } }
I20260807 18:39:06.843441 21465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:39:06.844558 21474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15826e199fd740ab8ab6e838b509332d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15826e199fd740ab8ab6e838b509332d" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33409 } } }
I20260807 18:39:06.844699 21474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [sys.catalog]: This master's current role is: LEADER
I20260807 18:39:06.845116 21476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15826e199fd740ab8ab6e838b509332d. Latest consensus state: current_term: 1 leader_uuid: "15826e199fd740ab8ab6e838b509332d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15826e199fd740ab8ab6e838b509332d" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33409 } } }
I20260807 18:39:06.845324 21476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d [sys.catalog]: This master's current role is: LEADER
I20260807 18:39:06.845858 21488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:39:06.848724 21488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:39:06.854321 21488 catalog_manager.cc:1347] Generated new cluster ID: 5d7c757ff74c4576946bd988b16595ed
I20260807 18:39:06.854391 21488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:39:06.868055 21488 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:39:06.869377 21488 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:39:06.877566 21488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15826e199fd740ab8ab6e838b509332d: Generated new TSK 0
I20260807 18:39:06.878441 21488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:39:06.887840 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33409
--builtin_ntp_servers=127.6.138.84:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 18:39:07.006275 21501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:07.006572 21501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:07.006640 21501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:07.010674 21501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:07.010785 21501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:07.010887 21501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:39:07.015738 21501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:07.016924 21501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:07.018468 21501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:07.028340 21514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:07.028340 21508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:07.028340 21507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:07.028532 21501 server_base.cc:1034] running on GCE node
I20260807 18:39:07.029057 21501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:07.029722 21501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:07.030920 21501 hybrid_clock.cc:648] HybridClock initialized: now 1786127947030905 us; error 36 us; skew 500 ppm
I20260807 18:39:07.032909 21501 webserver.cc:460] Webserver started at http://127.6.138.65:42539/ using document root <none> and password file <none>
I20260807 18:39:07.033522 21501 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:07.033627 21501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:07.033871 21501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:07.036007 21501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "1571d59ea02b4d0baa5104a4ea14dddb"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "1265ca75f44e423949e6135f741a8e73"
server_key_iv: "c0fe132eed5243d0baa74a45ec5b5bb5"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.036665 21501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "1571d59ea02b4d0baa5104a4ea14dddb"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "1265ca75f44e423949e6135f741a8e73"
server_key_iv: "c0fe132eed5243d0baa74a45ec5b5bb5"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.040414 21501 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:39:07.043088 21525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.044553 21501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:39:07.044708 21501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "1571d59ea02b4d0baa5104a4ea14dddb"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "1265ca75f44e423949e6135f741a8e73"
server_key_iv: "c0fe132eed5243d0baa74a45ec5b5bb5"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.044831 21501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:07.064007 21501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:07.064914 21501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:07.065141 21501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:07.065788 21501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:07.066833 21501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:07.066929 21501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.067003 21501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:07.067055 21501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.076547 21501 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:46375
I20260807 18:39:07.076579 21700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:46375 every 8 connection(s)
I20260807 18:39:07.077517 21501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:39:07.085951 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 21501
I20260807 18:39:07.086097 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:39:07.086416 6697 external_mini_cluster.cc:1428] Setting key 384fe05fde64681363cc39755e30a459
I20260807 18:39:07.087350 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:07.079475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:33807 (local address 127.6.138.126:33409)
0807 18:39:07.079638 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:07.079641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:07.080782 (+ 1141us) server_negotiation.cc:407] Connection header received
0807 18:39:07.082257 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:07.082261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:07.082322 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:07.082405 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:07.083326 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.083927 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.085178 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.085425 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.085557 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:07.086804 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:07.086808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:07.086810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:39:07.086825 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:07.086853 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:39:07.086862 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:07.086940 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:07.087136 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 18:39:07.087205 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:39:07.088450 21703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:07.079881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:33409 (local address 127.6.138.65:33807)
0807 18:39:07.080624 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:07.080667 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:07.082018 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:07.082582 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:07.082589 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:07.082677 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:07.083168 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:07.083181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.084085 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.084089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:07.085030 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:07.085050 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.085595 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.085599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:07.085774 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:07.086474 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:39:07.086476 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:07.086557 (+ 81us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:39:07.086643 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:07.086958 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:07.086969 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:07.086975 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:39:07.087128 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 18:39:07.087434 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":117,"threads_started":1}
I20260807 18:39:07.088634 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33409
--builtin_ntp_servers=127.6.138.84:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:39:07.089807 21701 heartbeater.cc:347] Connected to a master server at 127.6.138.126:33409
I20260807 18:39:07.090080 21701 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:07.090739 21701 heartbeater.cc:511] Master 127.6.138.126:33409 requested a full tablet report, sending...
I20260807 18:39:07.092489 21413 ts_manager.cc:194] Registered new tserver with Master: 1571d59ea02b4d0baa5104a4ea14dddb (127.6.138.65:46375)
I20260807 18:39:07.094179 21413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.65:33807
W20260807 18:39:07.206283 21706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:07.206535 21706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:07.206575 21706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:07.210460 21706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:07.210506 21706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:07.210557 21706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:39:07.215082 21706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:07.216045 21706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:07.217401 21706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:07.227488 21716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:07.227488 21720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:07.227492 21715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:07.227852 21706 server_base.cc:1034] running on GCE node
I20260807 18:39:07.228319 21706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:07.229048 21706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:07.230237 21706 hybrid_clock.cc:648] HybridClock initialized: now 1786127947230212 us; error 30 us; skew 500 ppm
I20260807 18:39:07.232851 21706 webserver.cc:460] Webserver started at http://127.6.138.66:39747/ using document root <none> and password file <none>
I20260807 18:39:07.233678 21706 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:07.233783 21706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:07.234050 21706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:07.237011 21706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "c313b1e107e34c89a40f9273d098c00b"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "449e1f57ea469eef1c3fb7d935532a61"
server_key_iv: "35be83e7d61b6b8f05d1b4ddf385e2c0"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.237756 21706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "c313b1e107e34c89a40f9273d098c00b"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "449e1f57ea469eef1c3fb7d935532a61"
server_key_iv: "35be83e7d61b6b8f05d1b4ddf385e2c0"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.242190 21706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:39:07.245052 21733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.246412 21706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:39:07.246573 21706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "c313b1e107e34c89a40f9273d098c00b"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "449e1f57ea469eef1c3fb7d935532a61"
server_key_iv: "35be83e7d61b6b8f05d1b4ddf385e2c0"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.246697 21706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:07.273025 21706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:07.273773 21706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:07.273950 21706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:07.274541 21706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:07.275501 21706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:07.275549 21706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.275588 21706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:07.275604 21706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.285053 21706 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:33111
I20260807 18:39:07.285121 21909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:33111 every 8 connection(s)
I20260807 18:39:07.286027 21706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:39:07.287734 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 21706
I20260807 18:39:07.287863 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:39:07.288319 6697 external_mini_cluster.cc:1428] Setting key 6eb4357dc06cb4c536159df31f79004b
I20260807 18:39:07.290745 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33409
--builtin_ntp_servers=127.6.138.84:34475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:39:07.297705 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:07.288184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:53687 (local address 127.6.138.126:33409)
0807 18:39:07.288358 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:07.288398 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:07.289817 (+ 1419us) server_negotiation.cc:407] Connection header received
0807 18:39:07.292018 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:07.292020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:07.292068 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:07.292136 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:07.293438 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.294039 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.295349 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.295601 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.295731 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:07.297094 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:07.297098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:07.297101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:39:07.297118 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:07.297149 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:39:07.297158 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:07.297250 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:07.297461 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 18:39:07.297532 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:39:07.298671 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:07.288680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:33409 (local address 127.6.138.66:53687)
0807 18:39:07.289617 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:07.289682 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:07.291745 (+ 2063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:07.292389 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:07.292403 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:07.292542 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:07.293268 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:07.293284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.294188 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.294192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:07.295192 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:07.295202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.295778 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.295782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:07.295973 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:07.296754 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:39:07.296756 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:07.296802 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:39:07.296877 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:07.297254 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:07.297266 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:07.297274 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:39:07.297443 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 18:39:07.297778 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":163,"threads_started":1}
I20260807 18:39:07.299968 21911 heartbeater.cc:347] Connected to a master server at 127.6.138.126:33409
I20260807 18:39:07.300305 21911 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:07.300918 21911 heartbeater.cc:511] Master 127.6.138.126:33409 requested a full tablet report, sending...
I20260807 18:39:07.302137 21413 ts_manager.cc:194] Registered new tserver with Master: c313b1e107e34c89a40f9273d098c00b (127.6.138.66:33111)
I20260807 18:39:07.303045 21413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.66:53687
W20260807 18:39:07.410805 21917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:07.411077 21917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:07.411113 21917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:07.415074 21917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:07.415135 21917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:07.415189 21917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:39:07.420058 21917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:07.421200 21917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:07.422565 21917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:07.432590 21930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:07.432677 21927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:07.432642 21917 server_base.cc:1034] running on GCE node
W20260807 18:39:07.432776 21926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:07.433267 21917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:07.433902 21917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:07.435105 21917 hybrid_clock.cc:648] HybridClock initialized: now 1786127947435091 us; error 29 us; skew 500 ppm
I20260807 18:39:07.437160 21917 webserver.cc:460] Webserver started at http://127.6.138.67:32875/ using document root <none> and password file <none>
I20260807 18:39:07.437768 21917 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:07.437831 21917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:07.438162 21917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:07.440352 21917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "24b3df1f9ce147778111f81a8861247d"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "9e9d341915a3ebb6edf38a91d8b880d6"
server_key_iv: "94bf96e4b3b569ebdd2d382e214aa081"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.440950 21917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "24b3df1f9ce147778111f81a8861247d"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "9e9d341915a3ebb6edf38a91d8b880d6"
server_key_iv: "94bf96e4b3b569ebdd2d382e214aa081"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.444967 21917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260807 18:39:07.447687 21938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.449106 21917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:39:07.449249 21917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "24b3df1f9ce147778111f81a8861247d"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "9e9d341915a3ebb6edf38a91d8b880d6"
server_key_iv: "94bf96e4b3b569ebdd2d382e214aa081"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.449373 21917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:07.464967 21917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:07.465759 21917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:07.465981 21917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:07.466650 21917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:07.468144 21917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:07.468215 21917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.468303 21917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:07.468336 21917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.477555 21917 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:46471
I20260807 18:39:07.477612 22107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:46471 every 8 connection(s)
I20260807 18:39:07.478520 21917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:39:07.478878 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 21917
I20260807 18:39:07.479002 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:39:07.479365 6697 external_mini_cluster.cc:1428] Setting key b4b71e333f89c19cc7d9a0bbf292aafc
I20260807 18:39:07.488083 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:07.480456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:53119 (local address 127.6.138.126:33409)
0807 18:39:07.480600 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:07.480605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:07.481925 (+ 1320us) server_negotiation.cc:407] Connection header received
0807 18:39:07.483352 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:07.483355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:07.483402 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:07.483467 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:07.484533 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.485058 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.486231 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.486444 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.486521 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:07.487683 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:07.487686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:07.487688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:39:07.487701 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:07.487716 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:39:07.487722 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:07.487776 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:07.487917 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 18:39:07.487966 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:39:07.489089 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:07.481036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:33409 (local address 127.6.138.67:53119)
0807 18:39:07.481770 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:07.481822 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:07.483154 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:07.483695 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:07.483706 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:07.483810 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:07.484392 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:07.484415 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.485183 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.485186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:07.486121 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:07.486131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:07.486597 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:07.486601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:07.486757 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:07.487438 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:39:07.487440 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:07.487480 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:39:07.487548 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:07.487793 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:07.487800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:07.487805 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:39:07.487903 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:39:07.488131 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":142,"threads_started":1}
I20260807 18:39:07.490346 22109 heartbeater.cc:347] Connected to a master server at 127.6.138.126:33409
I20260807 18:39:07.490577 22109 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:07.491132 22109 heartbeater.cc:511] Master 127.6.138.126:33409 requested a full tablet report, sending...
I20260807 18:39:07.492250 21413 ts_manager.cc:194] Registered new tserver with Master: 24b3df1f9ce147778111f81a8861247d (127.6.138.67:46471)
I20260807 18:39:07.493103 21413 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.67:53119
I20260807 18:39:07.493716 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:39:07.496613 21470 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:07.495716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.6.138.126:33409)
0807 18:39:07.495843 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:07.495847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:07.496028 (+ 181us) server_negotiation.cc:407] Connection header received
0807 18:39:07.496112 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:07.496114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:07.496152 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:07.496220 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:07.496453 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34688: BlockingRecv error: recv got EOF from 127.0.0.1:34688 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:39:07.496760 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 21501
I20260807 18:39:07.516641 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 21706
I20260807 18:39:07.539227 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 21917
I20260807 18:39:07.548978 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 21355
2026-08-07T18:39:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (943 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T18:39:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:39:07Z Disabled control of system clock
I20260807 18:39:07.587512 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:41199
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:41199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:39:07.708629 22124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:07.708918 22124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:07.708957 22124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:07.713034 22124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:39:07.713100 22124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:07.713116 22124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:07.713186 22124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:39:07.713215 22124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:39:07.718132 22124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:41199
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:41199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:07.719318 22124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:07.720816 22124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:07.729274 22136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:07.729290 22131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:07.729282 22134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:07.729571 22124 server_base.cc:1034] running on GCE node
I20260807 18:39:07.730142 22124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:07.731248 22124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:07.732437 22124 hybrid_clock.cc:648] HybridClock initialized: now 1786127947732413 us; error 34 us; skew 500 ppm
I20260807 18:39:07.734298 22124 webserver.cc:460] Webserver started at http://127.6.138.126:41571/ using document root <none> and password file <none>
I20260807 18:39:07.734891 22124 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:07.734988 22124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:07.735230 22124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:07.737375 22124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "0138321a8ad24a158cccdd8dd0704dce"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "6ef6614bb00ada73caccde87e4d2abda"
server_key_iv: "b88c7e601f80ddefa52d3063ca7fdc42"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.737970 22124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "0138321a8ad24a158cccdd8dd0704dce"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "6ef6614bb00ada73caccde87e4d2abda"
server_key_iv: "b88c7e601f80ddefa52d3063ca7fdc42"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.741693 22124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:39:07.744210 22143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.745517 22124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:39:07.745673 22124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "0138321a8ad24a158cccdd8dd0704dce"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "6ef6614bb00ada73caccde87e4d2abda"
server_key_iv: "b88c7e601f80ddefa52d3063ca7fdc42"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.745793 22124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:07.750519 22124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:07.751161 22124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:07.751356 22124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:07.758520 22124 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:41199
I20260807 18:39:07.758574 22233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:41199 every 8 connection(s)
I20260807 18:39:07.759516 22124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:39:07.762854 22236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:39:07.764308 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 22124
I20260807 18:39:07.764427 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:39:07.764721 6697 external_mini_cluster.cc:1428] Setting key 44dc4b619a20f059e0e6f4adcef881f0
I20260807 18:39:07.769536 22236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce: Bootstrap starting.
I20260807 18:39:07.770717 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:07.766262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.6.138.126:41199)
0807 18:39:07.766985 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:07.767004 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:07.767041 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:39:07.767727 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:07.767744 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:07.768118 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:07.768536 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:07.769423 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:07.769428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:07.769441 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:39:07.769452 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:07.769510 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:39:07.769527 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:07.769684 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:07.769760 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 18:39:07.770013 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":137,"threads_started":1}
I20260807 18:39:07.772588 22236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce: Neither blocks nor log segments found. Creating new log.
I20260807 18:39:07.773684 22236 log.cc:824] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce: Log is configured to *not* fsync() on all Append() calls
I20260807 18:39:07.775801 22236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce: No bootstrap required, opened a new log
I20260807 18:39:07.779954 22236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0138321a8ad24a158cccdd8dd0704dce" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41199 } }
I20260807 18:39:07.780201 22236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:39:07.780238 22236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0138321a8ad24a158cccdd8dd0704dce, State: Initialized, Role: FOLLOWER
I20260807 18:39:07.780704 22236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0138321a8ad24a158cccdd8dd0704dce" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41199 } }
I20260807 18:39:07.780830 22236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:39:07.780877 22236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:39:07.780961 22236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:39:07.781913 22236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0138321a8ad24a158cccdd8dd0704dce" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41199 } }
I20260807 18:39:07.782225 22236 leader_election.cc:302] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0138321a8ad24a158cccdd8dd0704dce; no voters:
I20260807 18:39:07.782519 22236 leader_election.cc:288] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:39:07.782667 22246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:39:07.782927 22246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [term 1 LEADER]: Becoming Leader. State: Replica: 0138321a8ad24a158cccdd8dd0704dce, State: Running, Role: LEADER
I20260807 18:39:07.783273 22246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0138321a8ad24a158cccdd8dd0704dce" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41199 } }
I20260807 18:39:07.783921 22236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:39:07.784916 22247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0138321a8ad24a158cccdd8dd0704dce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0138321a8ad24a158cccdd8dd0704dce" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41199 } } }
I20260807 18:39:07.785058 22247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [sys.catalog]: This master's current role is: LEADER
I20260807 18:39:07.785392 22249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0138321a8ad24a158cccdd8dd0704dce. Latest consensus state: current_term: 1 leader_uuid: "0138321a8ad24a158cccdd8dd0704dce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0138321a8ad24a158cccdd8dd0704dce" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41199 } } }
I20260807 18:39:07.785483 22258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:39:07.785490 22249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce [sys.catalog]: This master's current role is: LEADER
I20260807 18:39:07.788455 22258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:39:07.794433 22258 catalog_manager.cc:1347] Generated new cluster ID: 22961aadcec14ea9ae25a14082db0f1a
I20260807 18:39:07.794514 22258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:39:07.821847 22258 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:39:07.822233 22258 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:39:07.830726 22258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0138321a8ad24a158cccdd8dd0704dce: Generated new TSK 0
I20260807 18:39:07.831468 22258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:39:07.840988 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41199
--builtin_ntp_servers=127.6.138.84:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:39:07.958621 22277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:07.958904 22277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:07.958945 22277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:07.963011 22277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:07.963083 22277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:07.963142 22277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:39:07.967818 22277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:07.969035 22277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:07.970425 22277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:07.981170 22288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:07.981213 22292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:07.981262 22277 server_base.cc:1034] running on GCE node
W20260807 18:39:07.981170 22285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:07.981802 22277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:07.982649 22277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:07.983865 22277 hybrid_clock.cc:648] HybridClock initialized: now 1786127947983839 us; error 34 us; skew 500 ppm
I20260807 18:39:07.986291 22277 webserver.cc:460] Webserver started at http://127.6.138.65:37307/ using document root <none> and password file <none>
I20260807 18:39:07.986984 22277 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:07.987099 22277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:07.987366 22277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:07.989809 22277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "c29b1edebade4dbfacde33676d41db12"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "21d2b3d7ab530204913c4b091774b9a1"
server_key_iv: "c15a9fb57bfdc241eff3e24dab300356"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.990460 22277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "c29b1edebade4dbfacde33676d41db12"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "21d2b3d7ab530204913c4b091774b9a1"
server_key_iv: "c15a9fb57bfdc241eff3e24dab300356"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.994645 22277 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:39:07.997686 22301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:07.999117 22277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:39:07.999276 22277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "c29b1edebade4dbfacde33676d41db12"
format_stamp: "Formatted at 2026-08-07 18:39:07 on dist-test-slave-2d19"
server_key: "21d2b3d7ab530204913c4b091774b9a1"
server_key_iv: "c15a9fb57bfdc241eff3e24dab300356"
server_key_version: "encryptionkey@0"
I20260807 18:39:07.999418 22277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:08.006181 22277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:08.006876 22277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:08.007093 22277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:08.007702 22277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:08.008854 22277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:08.008939 22277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:08.009017 22277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:08.009075 22277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:08.018798 22277 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:34893
I20260807 18:39:08.018821 22482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:34893 every 8 connection(s)
I20260807 18:39:08.019868 22277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:39:08.026484 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:08.021866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:49209 (local address 127.6.138.126:41199)
0807 18:39:08.022023 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:08.022026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:08.023193 (+ 1167us) server_negotiation.cc:407] Connection header received
0807 18:39:08.024667 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:08.024671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:08.024721 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:08.024827 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:08.025971 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:08.025975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:08.025977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:39:08.025990 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:08.026021 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:39:08.026029 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:08.026136 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:08.026289 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 18:39:08.026340 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:39:08.027382 22487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:08.022333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:41199 (local address 127.6.138.65:49209)
0807 18:39:08.023021 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:08.023081 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:08.024428 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:08.025003 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:08.025011 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:08.025105 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:08.025707 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:39:08.025709 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:08.025749 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:39:08.025818 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:08.026129 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:08.026139 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:08.026148 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:39:08.026337 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 18:39:08.026586 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":129,"threads_started":1}
I20260807 18:39:08.028872 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 22277
I20260807 18:39:08.029006 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:39:08.029076 22485 heartbeater.cc:347] Connected to a master server at 127.6.138.126:41199
I20260807 18:39:08.029315 6697 external_mini_cluster.cc:1428] Setting key 0bf899fd8179282ebb1661233d5e938b
I20260807 18:39:08.029315 22485 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:08.029736 22485 heartbeater.cc:511] Master 127.6.138.126:41199 requested a full tablet report, sending...
I20260807 18:39:08.031581 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41199
--builtin_ntp_servers=127.6.138.84:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:39:08.031615 22176 ts_manager.cc:194] Registered new tserver with Master: c29b1edebade4dbfacde33676d41db12 (127.6.138.65:34893)
W20260807 18:39:08.149550 22488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:08.149811 22488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:08.149852 22488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:08.154100 22488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:08.154160 22488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:08.154217 22488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:39:08.159938 22488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:08.161176 22488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:08.162690 22488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:08.172854 22499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:08.173025 22488 server_base.cc:1034] running on GCE node
W20260807 18:39:08.172873 22495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:08.172850 22494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:08.173600 22488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:08.174310 22488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:08.175503 22488 hybrid_clock.cc:648] HybridClock initialized: now 1786127948175470 us; error 46 us; skew 500 ppm
I20260807 18:39:08.177556 22488 webserver.cc:460] Webserver started at http://127.6.138.66:35699/ using document root <none> and password file <none>
I20260807 18:39:08.178190 22488 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:08.178293 22488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:08.178562 22488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:08.180799 22488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "2758088b9a97426e950d3c2a1fd68466"
format_stamp: "Formatted at 2026-08-07 18:39:08 on dist-test-slave-2d19"
server_key: "367b58ec08388533be18901c1be3f14a"
server_key_iv: "5a45a3abfffd07142cc59ec6d460d387"
server_key_version: "encryptionkey@0"
I20260807 18:39:08.181396 22488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "2758088b9a97426e950d3c2a1fd68466"
format_stamp: "Formatted at 2026-08-07 18:39:08 on dist-test-slave-2d19"
server_key: "367b58ec08388533be18901c1be3f14a"
server_key_iv: "5a45a3abfffd07142cc59ec6d460d387"
server_key_version: "encryptionkey@0"
I20260807 18:39:08.185724 22488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:39:08.188621 22509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:08.189956 22488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:39:08.190147 22488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "2758088b9a97426e950d3c2a1fd68466"
format_stamp: "Formatted at 2026-08-07 18:39:08 on dist-test-slave-2d19"
server_key: "367b58ec08388533be18901c1be3f14a"
server_key_iv: "5a45a3abfffd07142cc59ec6d460d387"
server_key_version: "encryptionkey@0"
I20260807 18:39:08.190280 22488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:08.195096 22488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:08.195730 22488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:08.195917 22488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:08.196528 22488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:08.197463 22488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:08.197535 22488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:08.197629 22488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:08.197681 22488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:08.207207 22488 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:44751
I20260807 18:39:08.207239 22690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:44751 every 8 connection(s)
I20260807 18:39:08.208112 22488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:39:08.209592 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 22488
I20260807 18:39:08.209748 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:39:08.210177 6697 external_mini_cluster.cc:1428] Setting key 1c5172c62212af199432ba3631c9db60
I20260807 18:39:08.212702 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41199
--builtin_ntp_servers=127.6.138.84:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:39:08.215454 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:08.210165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:34205 (local address 127.6.138.126:41199)
0807 18:39:08.210333 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:08.210338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:08.211470 (+ 1132us) server_negotiation.cc:407] Connection header received
0807 18:39:08.213493 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:08.213495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:08.213532 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:08.213613 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:08.214935 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:08.214941 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:08.214946 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:39:08.214963 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:08.214992 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:39:08.215004 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:08.215105 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:08.215230 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 18:39:08.215280 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:39:08.216472 22693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:08.210596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:41199 (local address 127.6.138.66:34205)
0807 18:39:08.211312 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:08.211355 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:08.213230 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:08.213806 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:08.213818 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:08.213948 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:08.214564 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:39:08.214568 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:08.214630 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:39:08.214734 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:08.215085 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:08.215091 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:08.215096 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:39:08.215235 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 18:39:08.215450 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260807 18:39:08.217674 22691 heartbeater.cc:347] Connected to a master server at 127.6.138.126:41199
I20260807 18:39:08.217937 22691 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:08.218458 22691 heartbeater.cc:511] Master 127.6.138.126:41199 requested a full tablet report, sending...
I20260807 18:39:08.219717 22176 ts_manager.cc:194] Registered new tserver with Master: 2758088b9a97426e950d3c2a1fd68466 (127.6.138.66:44751)
W20260807 18:39:08.341686 22695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:08.341952 22695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:08.342028 22695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:08.346127 22695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:08.346192 22695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:08.346280 22695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:39:08.351428 22695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:08.352684 22695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:08.354107 22695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:08.364301 22707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:08.364323 22703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:08.364323 22705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:08.364369 22695 server_base.cc:1034] running on GCE node
I20260807 18:39:08.364907 22695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:08.365675 22695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:08.366880 22695 hybrid_clock.cc:648] HybridClock initialized: now 1786127948366834 us; error 50 us; skew 500 ppm
I20260807 18:39:08.369316 22695 webserver.cc:460] Webserver started at http://127.6.138.67:46393/ using document root <none> and password file <none>
I20260807 18:39:08.370049 22695 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:08.370131 22695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:08.370393 22695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:08.372808 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "c6ea32dda86245f0a945280d8b075e84"
format_stamp: "Formatted at 2026-08-07 18:39:08 on dist-test-slave-2d19"
server_key: "bafffb2f78c8132b6879c1caca4af937"
server_key_iv: "fcf3e82b0d7b9b700001add552028116"
server_key_version: "encryptionkey@0"
I20260807 18:39:08.373457 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "c6ea32dda86245f0a945280d8b075e84"
format_stamp: "Formatted at 2026-08-07 18:39:08 on dist-test-slave-2d19"
server_key: "bafffb2f78c8132b6879c1caca4af937"
server_key_iv: "fcf3e82b0d7b9b700001add552028116"
server_key_version: "encryptionkey@0"
I20260807 18:39:08.377652 22695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:39:08.380431 22715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:08.381680 22695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:39:08.381822 22695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "c6ea32dda86245f0a945280d8b075e84"
format_stamp: "Formatted at 2026-08-07 18:39:08 on dist-test-slave-2d19"
server_key: "bafffb2f78c8132b6879c1caca4af937"
server_key_iv: "fcf3e82b0d7b9b700001add552028116"
server_key_version: "encryptionkey@0"
I20260807 18:39:08.381944 22695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:08.386866 22695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:08.387496 22695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:08.387678 22695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:08.388237 22695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:08.389237 22695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:08.389308 22695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:08.389377 22695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:08.389432 22695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:08.398890 22695 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:45619
I20260807 18:39:08.398909 22896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:45619 every 8 connection(s)
I20260807 18:39:08.399865 22695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:39:08.402083 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 22695
I20260807 18:39:08.402192 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:39:08.402518 6697 external_mini_cluster.cc:1428] Setting key 90d5d10552e239014253ebe0e060d31d
I20260807 18:39:08.406872 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:08.401793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:34227 (local address 127.6.138.126:41199)
0807 18:39:08.401936 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:08.401940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:08.403363 (+ 1423us) server_negotiation.cc:407] Connection header received
0807 18:39:08.405098 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:08.405102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:08.405145 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:08.405220 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:08.406356 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:08.406360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:08.406363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:39:08.406378 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:08.406404 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:39:08.406412 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:08.406525 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:08.406685 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 18:39:08.406733 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:39:08.407791 22901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:08.402328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:41199 (local address 127.6.138.67:34227)
0807 18:39:08.403220 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:08.403264 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:08.404836 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:08.405407 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:08.405414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:08.405515 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:08.406083 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:39:08.406085 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:08.406136 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:39:08.406209 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:08.406523 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:08.406535 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:08.406543 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:39:08.406734 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 18:39:08.406990 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":151,"threads_started":1}
I20260807 18:39:08.408964 22898 heartbeater.cc:347] Connected to a master server at 127.6.138.126:41199
I20260807 18:39:08.409226 22898 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:08.409688 22898 heartbeater.cc:511] Master 127.6.138.126:41199 requested a full tablet report, sending...
I20260807 18:39:08.410778 22176 ts_manager.cc:194] Registered new tserver with Master: c6ea32dda86245f0a945280d8b075e84 (127.6.138.67:45619)
I20260807 18:39:08.416669 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:39:08.419600 22242 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:39:08.418749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.6.138.126:41199)
0807 18:39:08.418870 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:08.418874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:08.419078 (+ 204us) server_negotiation.cc:407] Connection header received
0807 18:39:08.419150 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:08.419152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:08.419189 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:08.419269 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:08.419491 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56540: BlockingRecv error: recv got EOF from 127.0.0.1:56540 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:39:08.419847 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 22277
I20260807 18:39:08.436548 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 22488
I20260807 18:39:08.448182 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 22695
I20260807 18:39:08.473721 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 22124
2026-08-07T18:39:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (925 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:39:08 dist-test-slave-2d19 krb5kdc[22915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:39:08 dist-test-slave-2d19 krb5kdc[22915](info): set up 2 sockets
Aug 07 18:39:08 dist-test-slave-2d19 krb5kdc[22915](info): commencing operation
krb5kdc: starting...
W20260807 18:39:09.747519 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:39:09 dist-test-slave-2d19 krb5kdc[22915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:39:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:39:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:39:09.884152 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:40141
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:40141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:39:09.999662 22932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:09.999939 22932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:09.999981 22932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:10.004148 22932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:39:10.004218 22932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:10.004238 22932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:10.004311 22932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:39:10.004344 22932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:39:10.010377 22932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:40141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:40141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:10.011744 22932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:10.013025 22932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:10.021667 22944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:10.021744 22947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:10.021876 22932 server_base.cc:1034] running on GCE node
W20260807 18:39:10.021750 22950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:10.022806 22932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:10.024304 22932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:10.025532 22932 hybrid_clock.cc:648] HybridClock initialized: now 1786127950025517 us; error 39 us; skew 500 ppm
Aug 07 18:39:10 dist-test-slave-2d19 krb5kdc[22915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127950, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:39:10.028803 22932 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:39:10.029933 22932 webserver.cc:460] Webserver started at http://127.6.138.126:36787/ using document root <none> and password file <none>
I20260807 18:39:10.030625 22932 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:10.030740 22932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:10.031040 22932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:10.033529 22932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "40813453851545eba58ff7c2a2317da1"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "147eceba18199387db0a8a23d8dad492"
server_key_iv: "5fe71095920c3ff7ed433528b6abf280"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.034207 22932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "40813453851545eba58ff7c2a2317da1"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "147eceba18199387db0a8a23d8dad492"
server_key_iv: "5fe71095920c3ff7ed433528b6abf280"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.038596 22932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:39:10.041401 22960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.042785 22932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:39:10.042994 22932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "40813453851545eba58ff7c2a2317da1"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "147eceba18199387db0a8a23d8dad492"
server_key_iv: "5fe71095920c3ff7ed433528b6abf280"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.043134 22932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:10.065374 22932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:10.068874 22932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:10.069114 22932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:10.076184 22932 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:40141
I20260807 18:39:10.076511 23053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:40141 every 8 connection(s)
I20260807 18:39:10.077275 22932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:39:10.080204 23054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:39:10.081961 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 22932
I20260807 18:39:10.082098 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:39:10.082388 6697 external_mini_cluster.cc:1428] Setting key 3e54e4903233b9adf120a009f2f0feb8
I20260807 18:39:10.087011 23054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1: Bootstrap starting.
I20260807 18:39:10.089668 23054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1: Neither blocks nor log segments found. Creating new log.
I20260807 18:39:10.090615 23054 log.cc:824] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:39:10 dist-test-slave-2d19 krb5kdc[22915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:39:10.092860 23054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1: No bootstrap required, opened a new log
I20260807 18:39:10.096237 23054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40813453851545eba58ff7c2a2317da1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40141 } }
I20260807 18:39:10.096558 23054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:39:10.096616 23054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40813453851545eba58ff7c2a2317da1, State: Initialized, Role: FOLLOWER
I20260807 18:39:10.097059 23054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40813453851545eba58ff7c2a2317da1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40141 } }
I20260807 18:39:10.097226 23054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:39:10.097319 23054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:39:10.097445 23054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:39:10.097999 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.083842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.6.138.126:40141)
0807 18:39:10.084375 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:10.084400 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:10.084439 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:39:10.085133 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:10.085151 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:10.085515 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:10.085832 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:10.086788 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.088053 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.089262 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.089879 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.090023 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.092879 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:10.092884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:10.092898 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:39:10.092959 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:10.094729 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.095196 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.095202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.095206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.095319 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.095623 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.095627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.095628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.096030 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:10.096246 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:10.096573 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:39:10.097072 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":149,"threads_started":1}
I20260807 18:39:10.098483 23054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40813453851545eba58ff7c2a2317da1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40141 } }
I20260807 18:39:10.098830 23054 leader_election.cc:302] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40813453851545eba58ff7c2a2317da1; no voters:
I20260807 18:39:10.099191 23054 leader_election.cc:288] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:39:10.099277 23066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:39:10.099481 23066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [term 1 LEADER]: Becoming Leader. State: Replica: 40813453851545eba58ff7c2a2317da1, State: Running, Role: LEADER
I20260807 18:39:10.099787 23066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40813453851545eba58ff7c2a2317da1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40141 } }
I20260807 18:39:10.100567 23054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:39:10.101689 23067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40813453851545eba58ff7c2a2317da1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40813453851545eba58ff7c2a2317da1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40141 } } }
I20260807 18:39:10.101732 23068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40813453851545eba58ff7c2a2317da1. Latest consensus state: current_term: 1 leader_uuid: "40813453851545eba58ff7c2a2317da1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40813453851545eba58ff7c2a2317da1" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40141 } } }
I20260807 18:39:10.101815 23067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:39:10.101843 23068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:39:10.102315 23078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:39:10.105867 23078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:39:10.111146 23078 catalog_manager.cc:1347] Generated new cluster ID: 269f3e5bbe484b3d903407503dd7e04c
I20260807 18:39:10.111224 23078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:39:10.118927 23078 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:39:10.120195 23078 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:39:10.134099 23078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1: Generated new TSK 0
I20260807 18:39:10.134977 23078 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:39:10.179517 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40141
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:39:10.321135 23105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:10.321384 23105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:10.321426 23105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:10.325410 23105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:10.325477 23105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:10.325536 23105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:39:10.331076 23105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:10.332170 23105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:10.333272 23105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:10.343112 23113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:10.343112 23114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:10.343144 23116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:10.343325 23105 server_base.cc:1034] running on GCE node
I20260807 18:39:10.343832 23105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:10.344579 23105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:10.345747 23105 hybrid_clock.cc:648] HybridClock initialized: now 1786127950345733 us; error 26 us; skew 500 ppm
Aug 07 18:39:10 dist-test-slave-2d19 krb5kdc[22915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127950, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:39:10.354027 23105 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:39:10.355118 23105 webserver.cc:460] Webserver started at http://127.6.138.65:42059/ using document root <none> and password file <none>
I20260807 18:39:10.355777 23105 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:10.355878 23105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:10.356140 23105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:10.358482 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "93ef93aa3e1546e8a8be81e5ced6cce3"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "931d051f93291cadd9bf7419a9a15340"
server_key_iv: "bc2c79d30024dbe5c6af5adc779ab696"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.359117 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "93ef93aa3e1546e8a8be81e5ced6cce3"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "931d051f93291cadd9bf7419a9a15340"
server_key_iv: "bc2c79d30024dbe5c6af5adc779ab696"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.363232 23105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 18:39:10.366102 23129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.367511 23105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:39:10.367682 23105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "93ef93aa3e1546e8a8be81e5ced6cce3"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "931d051f93291cadd9bf7419a9a15340"
server_key_iv: "bc2c79d30024dbe5c6af5adc779ab696"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.367813 23105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:10.391297 23105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:10.395041 23105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:10.395275 23105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:10.395959 23105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:10.396915 23105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:10.396992 23105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.397060 23105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:10.397111 23105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.406859 23105 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:33781
I20260807 18:39:10.406883 23301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:33781 every 8 connection(s)
I20260807 18:39:10.407771 23105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:39:10.409447 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 23105
I20260807 18:39:10.409574 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:39:10.409912 6697 external_mini_cluster.cc:1428] Setting key b9372f35b9033687f3955e33838b796a
Aug 07 18:39:10 dist-test-slave-2d19 krb5kdc[22915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127950, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:39:10.422670 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.411391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:55635 (local address 127.6.138.126:40141)
0807 18:39:10.411509 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:10.411512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:10.411524 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:39:10.412755 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:10.412762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:10.412827 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:10.412927 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:10.414210 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.414750 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.415934 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.416186 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.416319 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.418967 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:10.418970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:10.418972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:39:10.418998 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:10.420545 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.421090 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.421094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.421096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.421179 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.421551 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.421555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.421557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.421793 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:10.421944 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:10.422424 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 18:39:10.422540 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:39:10.423527 23307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.410368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:40141 (local address 127.6.138.65:55635)
0807 18:39:10.410990 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:10.411048 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:10.412542 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:10.413096 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:10.413103 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:10.413499 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:10.414049 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.414063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.414884 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.414888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.415797 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.415807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.416377 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.416381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.416572 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.417212 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:10.417229 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:10.418784 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:10.420674 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:10.420680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:10.420696 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:10.420974 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:10.421285 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:10.421288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:10.421290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:10.421438 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:10.421931 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:10.421937 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:10.422197 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 18:39:10.422371 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 18:39:10.422652 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":169,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
I20260807 18:39:10.424985 23302 heartbeater.cc:347] Connected to a master server at 127.6.138.126:40141
I20260807 18:39:10.425240 23302 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:10.425789 23302 heartbeater.cc:511] Master 127.6.138.126:40141 requested a full tablet report, sending...
I20260807 18:39:10.427757 22991 ts_manager.cc:194] Registered new tserver with Master: 93ef93aa3e1546e8a8be81e5ced6cce3 (127.6.138.65:33781)
I20260807 18:39:10.429466 22991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:55635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:39:10.444617 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40141
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:39:10.566612 23312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:10.566916 23312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:10.566962 23312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:10.571120 23312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:10.571204 23312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:10.571290 23312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:39:10.576910 23312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:10.578022 23312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:10.579075 23312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:10.588945 23328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:10.589042 23324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:10.589103 23312 server_base.cc:1034] running on GCE node
W20260807 18:39:10.589274 23325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:10.589671 23312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:10.590312 23312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:10.591506 23312 hybrid_clock.cc:648] HybridClock initialized: now 1786127950591492 us; error 25 us; skew 500 ppm
Aug 07 18:39:10 dist-test-slave-2d19 krb5kdc[22915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127950, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:39:10.594347 23312 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:39:10.595166 23312 webserver.cc:460] Webserver started at http://127.6.138.66:40155/ using document root <none> and password file <none>
I20260807 18:39:10.595695 23312 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:10.595754 23312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:10.595920 23312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:10.598003 23312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "432100e195494200991d69c542f66a9b"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "afcd44e03bfb54f857ddb80ff224357c"
server_key_iv: "a3efed03131eff75f275a94b856b60db"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.598520 23312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "432100e195494200991d69c542f66a9b"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "afcd44e03bfb54f857ddb80ff224357c"
server_key_iv: "a3efed03131eff75f275a94b856b60db"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.602559 23312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:39:10.604969 23338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.606168 23312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:39:10.606261 23312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "432100e195494200991d69c542f66a9b"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "afcd44e03bfb54f857ddb80ff224357c"
server_key_iv: "a3efed03131eff75f275a94b856b60db"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.606341 23312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:10.622689 23312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:10.625788 23312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:10.625944 23312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:10.626523 23312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:10.627322 23312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:10.627367 23312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.627408 23312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:10.627430 23312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.637033 23312 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:43029
I20260807 18:39:10.637117 23530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:43029 every 8 connection(s)
I20260807 18:39:10.637976 23312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:39:10.642772 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 23312
I20260807 18:39:10.642863 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:39:10.643149 6697 external_mini_cluster.cc:1428] Setting key 85e76eca11d17ed27df79225d80e1f56
Aug 07 18:39:10 dist-test-slave-2d19 krb5kdc[22915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127950, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:39:10.652711 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.639805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:37105 (local address 127.6.138.126:40141)
0807 18:39:10.639940 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:10.639943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:10.640841 (+ 898us) server_negotiation.cc:407] Connection header received
0807 18:39:10.642172 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:10.642175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:10.642224 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:10.642302 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:10.643604 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.644244 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.645535 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.645762 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.645876 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.648767 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:10.648771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:10.648773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:39:10.648797 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:10.650294 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.650953 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.650956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.650957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.651033 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.651477 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.651479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.651480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.651658 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:10.651734 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:10.652475 (+ 741us) server_negotiation.cc:299] Negotiation successful
0807 18:39:10.652585 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:39:10.653992 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.640147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:40141 (local address 127.6.138.66:37105)
0807 18:39:10.640684 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:10.640728 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:10.641963 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:10.642470 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:10.642477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:10.642889 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:10.643440 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.643455 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.644443 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.644447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.645396 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.645406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.645920 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.645924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.646095 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.646792 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:10.646810 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:10.648593 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:10.650416 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:10.650425 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:10.650446 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:10.650846 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:10.651156 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:10.651163 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:10.651168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:10.651373 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:10.651773 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:10.651783 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:10.652153 (+ 370us) client_negotiation.cc:769] Sending connection context
0807 18:39:10.652417 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 18:39:10.652807 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:39:10.655190 23531 heartbeater.cc:347] Connected to a master server at 127.6.138.126:40141
I20260807 18:39:10.655444 23531 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:10.656095 23531 heartbeater.cc:511] Master 127.6.138.126:40141 requested a full tablet report, sending...
I20260807 18:39:10.657181 22991 ts_manager.cc:194] Registered new tserver with Master: 432100e195494200991d69c542f66a9b (127.6.138.66:43029)
I20260807 18:39:10.657778 22991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:37105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:39:10.676384 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40141
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:39:10.792632 23536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:10.792902 23536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:10.792948 23536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:10.796922 23536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:10.796990 23536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:10.797047 23536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:39:10.802412 23536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:10.803535 23536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:10.804643 23536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:10.814682 23549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:10.814682 23547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:10.814702 23551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:10.815162 23536 server_base.cc:1034] running on GCE node
I20260807 18:39:10.815587 23536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:10.816315 23536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:10.817517 23536 hybrid_clock.cc:648] HybridClock initialized: now 1786127950817499 us; error 33 us; skew 500 ppm
Aug 07 18:39:10 dist-test-slave-2d19 krb5kdc[22915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127950, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:39:10.820492 23536 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:39:10.821400 23536 webserver.cc:460] Webserver started at http://127.6.138.67:46879/ using document root <none> and password file <none>
I20260807 18:39:10.821995 23536 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:10.822104 23536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:10.822355 23536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:39:10.824568 23536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "731cc7935e7b4cf5a2c15890bf3b6789"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "d75e286a1e77938be06a8a7d20ea88d6"
server_key_iv: "0f4f2194b0efef4a6d3b375268a281c9"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.825178 23536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "731cc7935e7b4cf5a2c15890bf3b6789"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "d75e286a1e77938be06a8a7d20ea88d6"
server_key_iv: "0f4f2194b0efef4a6d3b375268a281c9"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.829237 23536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:39:10.831976 23559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.833334 23536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:39:10.833482 23536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "731cc7935e7b4cf5a2c15890bf3b6789"
format_stamp: "Formatted at 2026-08-07 18:39:10 on dist-test-slave-2d19"
server_key: "d75e286a1e77938be06a8a7d20ea88d6"
server_key_iv: "0f4f2194b0efef4a6d3b375268a281c9"
server_key_version: "encryptionkey@0"
I20260807 18:39:10.833611 23536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:10.844398 23536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:10.847975 23536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:10.848201 23536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:10.848883 23536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:10.849746 23536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:10.849821 23536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.849898 23536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:10.849941 23536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:10.859369 23536 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:36947
I20260807 18:39:10.859393 23736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:36947 every 8 connection(s)
I20260807 18:39:10.860386 23536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:39:10.863986 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 23536
I20260807 18:39:10.864126 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:39:10.864550 6697 external_mini_cluster.cc:1428] Setting key fd740240345db9a1ca40a0570ac0a2fc
Aug 07 18:39:10 dist-test-slave-2d19 krb5kdc[22915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127950, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:39:10.875764 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.862300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:43285 (local address 127.6.138.126:40141)
0807 18:39:10.862481 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:10.862487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:10.863425 (+ 938us) server_negotiation.cc:407] Connection header received
0807 18:39:10.865296 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:10.865299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:10.865357 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:10.865452 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:10.867177 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.867724 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.869148 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.869377 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.869479 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.872231 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:10.872234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:10.872237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:39:10.872319 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:10.873829 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.874339 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.874342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.874343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.874422 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.874759 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.874762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.874763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.874940 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:10.875015 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:10.875522 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 18:39:10.875635 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:39:10.876714 23739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.862748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:40141 (local address 127.6.138.67:43285)
0807 18:39:10.863256 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:10.863303 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:10.865057 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:10.865835 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:10.865845 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:10.866345 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:10.867004 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.867021 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.867849 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.867852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.868865 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.868875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.869577 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.869584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.869736 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.870314 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:10.870331 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:10.872059 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:10.873950 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:10.873961 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:10.873975 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:10.874245 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:10.874509 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:10.874511 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:10.874512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:10.874674 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:10.875031 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:10.875037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:10.875307 (+ 270us) client_negotiation.cc:769] Sending connection context
0807 18:39:10.875474 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 18:39:10.875767 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":126,"threads_started":1}
I20260807 18:39:10.878165 23737 heartbeater.cc:347] Connected to a master server at 127.6.138.126:40141
I20260807 18:39:10.878434 23737 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:10.879019 23737 heartbeater.cc:511] Master 127.6.138.126:40141 requested a full tablet report, sending...
I20260807 18:39:10.879997 22989 ts_manager.cc:194] Registered new tserver with Master: 731cc7935e7b4cf5a2c15890bf3b6789 (127.6.138.67:36947)
I20260807 18:39:10.880620 22989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:43285
I20260807 18:39:10.890998 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:39:10.900537 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.893168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37598 (local address 127.6.138.126:40141)
0807 18:39:10.893330 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:10.893336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:10.893481 (+ 145us) server_negotiation.cc:407] Connection header received
0807 18:39:10.893594 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:10.893596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:10.893639 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:10.893708 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:10.894401 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.894905 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.895708 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.895917 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.895993 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.897335 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:10.897338 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:10.897340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:39:10.897364 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:10.898841 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.899273 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.899275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.899277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.899353 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:10.899643 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:10.899645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:10.899646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:10.899859 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:10.899965 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:10.900217 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 18:39:10.900366 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:39:10.903849 22989 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:39:10.906966 22989 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:39:10.922334 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.916589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33630 (local address 127.6.138.66:43029)
0807 18:39:10.916935 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:10.916947 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:10.917729 (+ 782us) server_negotiation.cc:407] Connection header received
0807 18:39:10.917954 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:10.917961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:10.918141 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:10.918300 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:39:10.918484 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.919657 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.921083 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.921742 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.921853 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.922005 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:10.922117 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 18:39:10.922206 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":113,"threads_started":1}
I20260807 18:39:10.922313 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.916084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.6.138.67:36947)
0807 18:39:10.916589 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:10.916604 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:10.917103 (+ 499us) server_negotiation.cc:407] Connection header received
0807 18:39:10.917360 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:10.917368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:10.917604 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:10.917743 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:39:10.918205 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.919078 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.920639 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.921495 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.921617 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.921765 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:10.922034 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 18:39:10.922159 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":210,"threads_started":1}
I20260807 18:39:10.922377 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.917106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:43029 (local address 127.0.0.1:33630)
0807 18:39:10.917636 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:10.917648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:10.917798 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:10.918273 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:10.918276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:10.918294 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:39:10.918384 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.918389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.919808 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.919811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.920961 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.920970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.921855 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.921858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.921955 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.921979 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:39:10.922069 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:39:10.922201 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":76,"thread_start_us":90,"threads_started":1}
I20260807 18:39:10.922747 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.916680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:33781 (local address 127.0.0.1:55364)
0807 18:39:10.917314 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:10.917328 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:10.917428 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:10.918027 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:10.918031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:10.918050 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:39:10.918185 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.918196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.920010 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.920021 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.921217 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.921230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.922327 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.922330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.922430 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.922460 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:39:10.922514 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 18:39:10.922579 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
I20260807 18:39:10.922786 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.915909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:36947 (local address 127.0.0.1:46686)
0807 18:39:10.916862 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:10.916977 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:10.917141 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:10.917964 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:10.917969 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:10.917988 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:39:10.918087 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.918091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.919214 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.919219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.920465 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:10.920477 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.921644 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.921648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:10.921759 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.921822 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 18:39:10.922554 (+ 732us) client_negotiation.cc:240] Negotiation successful
0807 18:39:10.922639 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":29,"thread_start_us":94,"threads_started":1}
I20260807 18:39:10.923493 23759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:10.916939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55364 (local address 127.6.138.65:33781)
0807 18:39:10.917305 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:10.917319 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:10.917397 (+ 78us) server_negotiation.cc:407] Connection header received
0807 18:39:10.917540 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:10.917547 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:10.917828 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:10.918330 (+ 502us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:39:10.918483 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.919657 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.921369 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:10.922197 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:10.923116 (+ 919us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:10.923267 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:10.923310 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 18:39:10.923382 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":89,"threads_started":1}
I20260807 18:39:10.926151 23202 tablet_service.cc:1467] Processing CreateTablet for tablet fba20c39ebb94202b01516739247dcea (DEFAULT_TABLE table=test-table [id=100e57ca959d43f8b5a690f64e4cdfc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:39:10.926331 23414 tablet_service.cc:1467] Processing CreateTablet for tablet fba20c39ebb94202b01516739247dcea (DEFAULT_TABLE table=test-table [id=100e57ca959d43f8b5a690f64e4cdfc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:39:10.926324 23632 tablet_service.cc:1467] Processing CreateTablet for tablet fba20c39ebb94202b01516739247dcea (DEFAULT_TABLE table=test-table [id=100e57ca959d43f8b5a690f64e4cdfc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:39:10.927284 23202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba20c39ebb94202b01516739247dcea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:39:10.927369 23632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba20c39ebb94202b01516739247dcea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:39:10.927362 23414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fba20c39ebb94202b01516739247dcea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:39:10.933818 23763 tablet_bootstrap.cc:492] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3: Bootstrap starting.
I20260807 18:39:10.935555 23767 tablet_bootstrap.cc:492] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: Bootstrap starting.
I20260807 18:39:10.936421 23763 tablet_bootstrap.cc:654] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3: Neither blocks nor log segments found. Creating new log.
I20260807 18:39:10.936769 23765 tablet_bootstrap.cc:492] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: Bootstrap starting.
I20260807 18:39:10.937403 23763 log.cc:824] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:39:10.938506 23767 tablet_bootstrap.cc:654] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: Neither blocks nor log segments found. Creating new log.
I20260807 18:39:10.939394 23763 tablet_bootstrap.cc:492] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3: No bootstrap required, opened a new log
I20260807 18:39:10.939420 23767 log.cc:824] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: Log is configured to *not* fsync() on all Append() calls
I20260807 18:39:10.939613 23763 ts_tablet_manager.cc:1397] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 18:39:10.939668 23765 tablet_bootstrap.cc:654] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: Neither blocks nor log segments found. Creating new log.
I20260807 18:39:10.940562 23765 log.cc:824] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:39:10.941562 23767 tablet_bootstrap.cc:492] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: No bootstrap required, opened a new log
I20260807 18:39:10.941756 23767 ts_tablet_manager.cc:1397] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:39:10.942626 23765 tablet_bootstrap.cc:492] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: No bootstrap required, opened a new log
I20260807 18:39:10.942814 23765 ts_tablet_manager.cc:1397] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:39:10.943954 23763 raft_consensus.cc:348] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } }
I20260807 18:39:10.944219 23763 raft_consensus.cc:374] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:39:10.944300 23763 raft_consensus.cc:729] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ef93aa3e1546e8a8be81e5ced6cce3, State: Initialized, Role: FOLLOWER
I20260807 18:39:10.944782 23763 consensus_queue.cc:260] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } }
I20260807 18:39:10.945495 23302 heartbeater.cc:503] Master 127.6.138.126:40141 was elected leader, sending a full tablet report...
I20260807 18:39:10.945852 23763 ts_tablet_manager.cc:1428] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:39:10.945719 23767 raft_consensus.cc:348] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } }
I20260807 18:39:10.946120 23767 raft_consensus.cc:374] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:39:10.946185 23767 raft_consensus.cc:729] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 731cc7935e7b4cf5a2c15890bf3b6789, State: Initialized, Role: FOLLOWER
I20260807 18:39:10.946642 23767 consensus_queue.cc:260] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } }
I20260807 18:39:10.946743 23765 raft_consensus.cc:348] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } }
I20260807 18:39:10.947012 23765 raft_consensus.cc:374] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:39:10.947077 23765 raft_consensus.cc:729] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432100e195494200991d69c542f66a9b, State: Initialized, Role: FOLLOWER
I20260807 18:39:10.947538 23765 consensus_queue.cc:260] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } }
I20260807 18:39:10.947698 23767 ts_tablet_manager.cc:1428] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:39:10.948292 23531 heartbeater.cc:503] Master 127.6.138.126:40141 was elected leader, sending a full tablet report...
I20260807 18:39:10.948524 23765 ts_tablet_manager.cc:1428] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:39:10.948925 23737 heartbeater.cc:503] Master 127.6.138.126:40141 was elected leader, sending a full tablet report...
I20260807 18:39:11.065222 23780 raft_consensus.cc:482] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:39:11.065446 23780 raft_consensus.cc:504] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } }
I20260807 18:39:11.066727 23780 leader_election.cc:288] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 731cc7935e7b4cf5a2c15890bf3b6789 (127.6.138.67:36947), 93ef93aa3e1546e8a8be81e5ced6cce3 (127.6.138.65:33781)
I20260807 18:39:11.070781 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:11.066981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:36947 (local address 127.6.138.66:52079)
0807 18:39:11.067174 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:11.067195 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:11.067319 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:11.067865 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:11.067869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:11.067916 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:39:11.068043 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:11.068049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.068878 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.068883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:11.069667 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:11.069677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.070336 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.070340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:11.070448 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:11.070481 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:39:11.070570 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:39:11.070645 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"spinlock_wait_cycles":7424}
I20260807 18:39:11.070782 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:11.067123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:52079 (local address 127.6.138.67:36947)
0807 18:39:11.067288 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:11.067293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:11.067310 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:39:11.067694 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:11.067699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:11.067755 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:11.067850 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:39:11.068163 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.068760 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.069794 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.070212 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.070344 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:11.070400 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:11.070587 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 18:39:11.070655 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:39:11.071215 23759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:11.067537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:36937 (local address 127.6.138.65:33781)
0807 18:39:11.067676 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:11.067681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:11.067856 (+ 175us) server_negotiation.cc:407] Connection header received
0807 18:39:11.068046 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:11.068049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:11.068106 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:11.068194 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:39:11.068489 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.068986 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.070016 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.070475 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.070601 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:11.070652 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:11.071020 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 18:39:11.071102 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:39:11.071527 23666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba20c39ebb94202b01516739247dcea" candidate_uuid: "432100e195494200991d69c542f66a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" is_pre_election: true
I20260807 18:39:11.071651 23781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:11.067345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:33781 (local address 127.6.138.66:36937)
0807 18:39:11.067746 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:11.067757 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:11.067928 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:11.068204 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:11.068207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:11.068223 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:39:11.068368 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:11.068374 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.069106 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.069111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:11.069883 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:11.069891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.070757 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.070760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:11.070874 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:11.070899 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:39:11.071394 (+ 495us) client_negotiation.cc:240] Negotiation successful
0807 18:39:11.071477 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"spinlock_wait_cycles":36608,"thread_start_us":187,"threads_started":1}
I20260807 18:39:11.071957 23666 raft_consensus.cc:2454] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 432100e195494200991d69c542f66a9b in term 0.
I20260807 18:39:11.072454 23229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba20c39ebb94202b01516739247dcea" candidate_uuid: "432100e195494200991d69c542f66a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" is_pre_election: true
I20260807 18:39:11.072474 23340 leader_election.cc:302] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 432100e195494200991d69c542f66a9b, 731cc7935e7b4cf5a2c15890bf3b6789; no voters:
I20260807 18:39:11.072762 23229 raft_consensus.cc:2454] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 432100e195494200991d69c542f66a9b in term 0.
I20260807 18:39:11.072780 23780 raft_consensus.cc:2781] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:39:11.072877 23780 raft_consensus.cc:482] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:39:11.072942 23780 raft_consensus.cc:3037] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:39:11.074124 23780 raft_consensus.cc:504] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } }
I20260807 18:39:11.074488 23780 leader_election.cc:288] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [CANDIDATE]: Term 1 election: Requested vote from peers 731cc7935e7b4cf5a2c15890bf3b6789 (127.6.138.67:36947), 93ef93aa3e1546e8a8be81e5ced6cce3 (127.6.138.65:33781)
I20260807 18:39:11.074925 23666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba20c39ebb94202b01516739247dcea" candidate_uuid: "432100e195494200991d69c542f66a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "731cc7935e7b4cf5a2c15890bf3b6789"
I20260807 18:39:11.074990 23229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fba20c39ebb94202b01516739247dcea" candidate_uuid: "432100e195494200991d69c542f66a9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3"
I20260807 18:39:11.075098 23666 raft_consensus.cc:3037] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:39:11.075153 23229 raft_consensus.cc:3037] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:39:11.076380 23666 raft_consensus.cc:2454] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 432100e195494200991d69c542f66a9b in term 1.
I20260807 18:39:11.076467 23229 raft_consensus.cc:2454] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 432100e195494200991d69c542f66a9b in term 1.
I20260807 18:39:11.076715 23340 leader_election.cc:302] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 432100e195494200991d69c542f66a9b, 731cc7935e7b4cf5a2c15890bf3b6789; no voters:
I20260807 18:39:11.076891 23780 raft_consensus.cc:2781] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:39:11.077157 23780 raft_consensus.cc:686] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 1 LEADER]: Becoming Leader. State: Replica: 432100e195494200991d69c542f66a9b, State: Running, Role: LEADER
I20260807 18:39:11.077512 23780 consensus_queue.cc:237] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } }
I20260807 18:39:11.080930 22991 catalog_manager.cc:5495] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 432100e195494200991d69c542f66a9b (127.6.138.66). New cstate: current_term: 1 leader_uuid: "432100e195494200991d69c542f66a9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "432100e195494200991d69c542f66a9b" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 43029 } health_report { overall_health: HEALTHY } } }
W20260807 18:39:11.111423 23738 tablet.cc:2369] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:39:11.117859 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:11.114029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33636 (local address 127.6.138.66:43029)
0807 18:39:11.114201 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:11.114207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:11.114222 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:39:11.114424 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:11.114427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:11.114488 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:11.114613 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:39:11.114873 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.115393 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.116317 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:11.116542 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:11.116659 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:11.116988 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:39:11.117432 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:39:11.117576 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:11.117646 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 18:39:11.117706 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:39:11.124207 23666 raft_consensus.cc:1261] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [term 1 FOLLOWER]: Refusing update from remote peer 432100e195494200991d69c542f66a9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:39:11.124219 23229 raft_consensus.cc:1261] T fba20c39ebb94202b01516739247dcea P 93ef93aa3e1546e8a8be81e5ced6cce3 [term 1 FOLLOWER]: Refusing update from remote peer 432100e195494200991d69c542f66a9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:39:11.125072 23780 consensus_queue.cc:1035] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "731cc7935e7b4cf5a2c15890bf3b6789" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:39:11.125349 23782 consensus_queue.cc:1035] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "93ef93aa3e1546e8a8be81e5ced6cce3" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 33781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:39:11.133248 23786 mvcc.cc:205] Tried to move back new op lower bound from 7315980087796404224 to 7315980087616331776. Current Snapshot: MvccSnapshot[applied={T|T < 7315980087796404224}]
I20260807 18:39:11.134622 23787 mvcc.cc:205] Tried to move back new op lower bound from 7315980087796404224 to 7315980087616331776. Current Snapshot: MvccSnapshot[applied={T|T < 7315980087796404224}]
I20260807 18:39:11.141880 22987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:39:11.142083 22987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:39:11.144565 22987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40813453851545eba58ff7c2a2317da1: Sending DeleteTablet for 3 replicas of tablet fba20c39ebb94202b01516739247dcea
I20260807 18:39:11.145430 23632 tablet_service.cc:1514] Processing DeleteTablet for tablet fba20c39ebb94202b01516739247dcea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:39:11 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:46686
I20260807 18:39:11.145490 23202 tablet_service.cc:1514] Processing DeleteTablet for tablet fba20c39ebb94202b01516739247dcea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:39:11 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:55364
I20260807 18:39:11.145529 23414 tablet_service.cc:1514] Processing DeleteTablet for tablet fba20c39ebb94202b01516739247dcea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:39:11 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:33630
I20260807 18:39:11.146039 23804 tablet_replica.cc:330] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: stopping tablet replica
I20260807 18:39:11.146363 23804 raft_consensus.cc:2229] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:39:11.146385 23803 tablet_replica.cc:330] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: stopping tablet replica
I20260807 18:39:11.146801 23804 raft_consensus.cc:2258] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:39:11.147056 23803 raft_consensus.cc:2229] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:39:11.147481 23803 raft_consensus.cc:2258] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:39:11.147639 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 23105
I20260807 18:39:11.148483 23804 ts_tablet_manager.cc:1905] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:39:11.151813 23804 ts_tablet_manager.cc:1918] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:39:11.151929 23804 log.cc:1196] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/fba20c39ebb94202b01516739247dcea
I20260807 18:39:11.152209 23804 ts_tablet_manager.cc:1939] T fba20c39ebb94202b01516739247dcea P 432100e195494200991d69c542f66a9b: Deleting consensus metadata
I20260807 18:39:11.155380 22965 catalog_manager.cc:4841] TS 432100e195494200991d69c542f66a9b (127.6.138.66:43029): tablet fba20c39ebb94202b01516739247dcea (table test-table [id=100e57ca959d43f8b5a690f64e4cdfc7]) successfully deleted
I20260807 18:39:11.159819 23803 ts_tablet_manager.cc:1905] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:39:11.163620 23803 ts_tablet_manager.cc:1918] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:39:11.163744 23803 log.cc:1196] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/fba20c39ebb94202b01516739247dcea
I20260807 18:39:11.164139 23803 ts_tablet_manager.cc:1939] T fba20c39ebb94202b01516739247dcea P 731cc7935e7b4cf5a2c15890bf3b6789: Deleting consensus metadata
W20260807 18:39:11.165535 22964 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:33781 (error 108)
I20260807 18:39:11.166123 22964 catalog_manager.cc:4841] TS 731cc7935e7b4cf5a2c15890bf3b6789 (127.6.138.67:36947): tablet fba20c39ebb94202b01516739247dcea (table test-table [id=100e57ca959d43f8b5a690f64e4cdfc7]) successfully deleted
I20260807 18:39:11.166668 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 23312
I20260807 18:39:11.166752 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:11.166443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:33781 (local address 127.0.0.1:55368)
0807 18:39:11.166580 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:11.166654 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:33781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 18:39:11.184419 22964 catalog_manager.cc:4584] TS 93ef93aa3e1546e8a8be81e5ced6cce3 (127.6.138.65:33781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fba20c39ebb94202b01516739247dcea: Network error: Client connection negotiation failed: client connection to 127.6.138.65:33781: connect: Connection refused (error 111)
I20260807 18:39:11.196240 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 23536
I20260807 18:39:11.211998 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 22932
2026-08-07T18:39:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2740 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:39:11 dist-test-slave-2d19 krb5kdc[23814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:39:11 dist-test-slave-2d19 krb5kdc[23814](info): set up 2 sockets
Aug 07 18:39:11 dist-test-slave-2d19 krb5kdc[23814](info): commencing operation
krb5kdc: starting...
W20260807 18:39:12.514647 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:39:12 dist-test-slave-2d19 krb5kdc[23814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:39:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:39:12Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.138.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.138.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/rangeradmin_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/rangeradmin_127.6.138.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.138.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.138.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/rangerlookup_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/rangerlookup_127.6.138.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.138.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.138.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/HTTP_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/HTTP_127.6.138.84@KRBTEST.COM.keytab.
I20260807 18:39:12.708055 6697 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPYlYkN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/postgres -l logfile start
2026-08-07 18:39:15.377 UTC [23867] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:39:15.377 UTC [23867] LOG: listening on IPv4 address "127.6.138.84", port 59181
2026-08-07 18:39:15.381 UTC [23867] LOG: listening on Unix socket "/tmp/.s.PGSQL.59181"
2026-08-07 18:39:15.404 UTC [23870] LOG: database system was shut down at 2026-08-07 18:39:13 UTC
2026-08-07 18:39:15.408 UTC [23867] LOG: database system is ready to accept connections
I20260807 18:39:16.587925 6697 mini_postgres.cc:96] Postgres bound to 59181
2026-08-07 18:39:16.597 UTC [23884] FATAL: database "slave" does not exist
127.6.138.84:59181 - accepting connections
I20260807 18:39:16.598347 6697 mini_ranger.cc:161] Starting Ranger...
I20260807 18:39:16.616221 6697 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:39:17.133592 6697 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:39:17.133719 6697 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-admin
2026-08-07 18:39:17,471 [I] DB FLAVOR :POSTGRES
2026-08-07 18:39:17,471 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:39:17,471 [I] Checking connection..
2026-08-07 18:39:17,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:39:17,738 [I] Checking connection passed.
2026-08-07 18:39:17,738 [I] --------- Verifying version history table ---------
2026-08-07 18:39:17,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:39:18,003 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:39:18,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:39:18,262 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:39:18,263 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:39:18,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:39:18.521 UTC [23979] WARNING: there is no transaction in progress
2026-08-07 18:39:18,528 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:39:18,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:39:18,792 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:39:18,793 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:39:18,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:39:19,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:39:19,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2d19.c.gcp-upstream.internal','N') ;"
2026-08-07 18:39:19,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:39:19,810 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:39:19,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:39:20,065 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:39:20,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:39:20,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:39:20,620 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:39:20,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:39:21.638 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.654 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.671 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.686 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.705 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.727 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.793 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.800 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.815 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.825 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.835 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.849 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.859 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.865 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.872 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:21.878 UTC [24169] WARNING: there is no transaction in progress
2026-08-07 18:39:22,537 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:39:22,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:39:22,788 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:39:22,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:39:23,059 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:39:23,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:39:23,336 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:39:23,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:39:23,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:39:23,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2d19.c.gcp-upstream.internal';"
2026-08-07 18:39:24,108 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:39:24,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:39:24,382 [I] DB_PATCHES have already been applied
I20260807 18:39:24.389025 6697 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPYlYkN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:39:24.389125 6697 mini_ranger.cc:193] Using host: 127.6.138.84
I20260807 18:39:24.394120 6697 mini_ranger.cc:227] Ranger admin URL: http://127.6.138.84:58941
Aug 07, 2026 6:39:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:39:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:39:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:39:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:39:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:39:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:39:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:39:25.751163 6697 mini_ranger.cc:160] Time spent starting Ranger: real 9.153s user 0.002s sys 0.012s
Aug 07, 2026 6:39:46 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:39:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 6:39:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:39:47 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:39:47 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:39:48.504510 6697 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.138.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.138.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/rangerkms_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/rangerkms_127.6.138.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.138.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.138.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.138.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/HTTP_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.138.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/HTTP_127.6.138.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:39:48 dist-test-slave-2d19 krb5kdc[23814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127988, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:39:48.620222 6697 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:39:48.633111 6697 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:39:49.554936 6697 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:39:49.555053 6697 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-kms
I20260807 18:39:49.555176 6697 mini_ranger_kms.cc:227] Using postgres at 127.6.138.84:59181
2026-08-07 18:39:49,949 [I] DB FLAVOR :POSTGRES
2026-08-07 18:39:49,950 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:39:49,950 [I] Checking connection
2026-08-07 18:39:49,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:39:50,199 [I] connection success
2026-08-07 18:39:50,199 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:39:50,199 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:39:50,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:39:50,480 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:39:50,480 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:39:50,480 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:39:50,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:59181/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:39:50,780 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:39:51.278702 6697 mini_ranger_kms.cc:310] Created kms service
I20260807 18:39:51.486495 6697 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:39:51.540666 6697 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:39:51 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127988, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:39:51.979118 6697 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:39:51.979283 6697 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:39:51.979311 6697 mini_ranger_kms.cc:240] Using host: 127.6.138.84
I20260807 18:39:51.984956 6697 mini_ranger_kms.cc:276] Ranger KMS PID: 24458
I20260807 18:39:51.985095 6697 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.138.84:60815
Aug 07, 2026 6:39:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:39:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:39:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:39:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:39:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:39:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 18:39:54 dist-test-slave-2d19 krb5kdc[23814](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127994, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.138.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:39:54.902535 6697 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.282s user 0.003s sys 0.288s
I20260807 18:39:54.902681 6697 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:39:54.902833 6697 mini_ranger_kms.cc:401] 127.6.138.84:60815/kms/v1/keys
Aug 07 18:39:55 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127994, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.138.84@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
Aug 07 18:39:55 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127994, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.138.84@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
Aug 07 18:39:58 dist-test-slave-2d19 krb5kdc[23814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:39:58.059888 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:40039
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:40039
--ranger_config_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:60815/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:39:58.186681 24536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:58.187033 24536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:58.187144 24536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:58.192884 24536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:39:58.192981 24536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:58.193018 24536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:58.193055 24536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:39:58.193089 24536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:39:58.198985 24536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.138.84:60815/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:40039
--ranger_config_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:40039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:58.200146 24536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:58.201231 24536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:58.210253 24547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:58.210284 24542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:58.210510 24536 server_base.cc:1034] running on GCE node
W20260807 18:39:58.210315 24551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:58.211272 24536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:58.212496 24536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:58.213688 24536 hybrid_clock.cc:648] HybridClock initialized: now 1786127998213661 us; error 43 us; skew 500 ppm
Aug 07 18:39:58 dist-test-slave-2d19 krb5kdc[23814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:39:58.219743 24536 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:39:58.220786 24536 webserver.cc:460] Webserver started at http://127.6.138.126:40779/ using document root <none> and password file <none>
I20260807 18:39:58.221398 24536 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:58.221485 24536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:58.221719 24536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:39:58 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:39:58.277931 24536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "989f320b06644238846be919893eeecc"
format_stamp: "Formatted at 2026-08-07 18:39:58 on dist-test-slave-2d19"
server_key: "2909962579425f9074a786d4827d9809"
server_key_iv: "57d054085d9778d07f6634c74223cf83"
server_key_version: "kuduclusterkey@0"
I20260807 18:39:58.278595 24536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "989f320b06644238846be919893eeecc"
format_stamp: "Formatted at 2026-08-07 18:39:58 on dist-test-slave-2d19"
server_key: "2909962579425f9074a786d4827d9809"
server_key_iv: "57d054085d9778d07f6634c74223cf83"
server_key_version: "kuduclusterkey@0"
I20260807 18:39:58.282734 24536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:39:58.297240 24565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:58.298801 24536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:39:58.298960 24536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "989f320b06644238846be919893eeecc"
format_stamp: "Formatted at 2026-08-07 18:39:58 on dist-test-slave-2d19"
server_key: "2909962579425f9074a786d4827d9809"
server_key_iv: "57d054085d9778d07f6634c74223cf83"
server_key_version: "kuduclusterkey@0"
I20260807 18:39:58.299108 24536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:58.312773 24536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:58.315866 24536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:58.316083 24536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:58.326066 24536 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:40039
I20260807 18:39:58.326114 24642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:40039 every 8 connection(s)
I20260807 18:39:58.327173 24536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:39:58.329222 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 24536
I20260807 18:39:58.329358 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:39:58.329674 6697 external_mini_cluster.cc:1428] Setting key 0323bc0f536875ba5e8dacfea857b223
I20260807 18:39:58.330425 24647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:39:58.338119 24647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc: Bootstrap starting.
I20260807 18:39:58.340987 24647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc: Neither blocks nor log segments found. Creating new log.
Aug 07 18:39:58 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:39:58.342206 24647 log.cc:824] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc: Log is configured to *not* fsync() on all Append() calls
I20260807 18:39:58.344713 24647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc: No bootstrap required, opened a new log
I20260807 18:39:58.348320 24647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989f320b06644238846be919893eeecc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40039 } }
I20260807 18:39:58.348652 24647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:39:58.348733 24647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989f320b06644238846be919893eeecc, State: Initialized, Role: FOLLOWER
I20260807 18:39:58.349246 24647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989f320b06644238846be919893eeecc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40039 } }
I20260807 18:39:58.349423 24647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:39:58.349506 24647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:39:58.349622 24647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:39:58.350003 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:58.331393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.6.138.126:40039)
0807 18:39:58.332013 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:58.332036 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:58.332090 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:39:58.333083 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:58.333109 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:58.333557 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:58.333997 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:58.334921 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:58.336613 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:58.337803 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:58.338527 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:58.338688 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:58.342129 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:58.342137 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:58.342156 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:39:58.342191 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:58.345371 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:58.346116 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:58.346122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:58.346128 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:58.346287 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:58.346774 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:58.346778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:58.346779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:58.347431 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:58.348296 (+ 865us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:58.348477 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 18:39:58.348771 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":223,"threads_started":1}
I20260807 18:39:58.350817 24647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989f320b06644238846be919893eeecc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40039 } }
I20260807 18:39:58.351189 24647 leader_election.cc:302] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 989f320b06644238846be919893eeecc; no voters:
I20260807 18:39:58.351601 24647 leader_election.cc:288] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:39:58.351713 24653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:39:58.351986 24653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [term 1 LEADER]: Becoming Leader. State: Replica: 989f320b06644238846be919893eeecc, State: Running, Role: LEADER
I20260807 18:39:58.352344 24653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989f320b06644238846be919893eeecc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40039 } }
I20260807 18:39:58.352988 24647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:39:58.353582 24653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 989f320b06644238846be919893eeecc. Latest consensus state: current_term: 1 leader_uuid: "989f320b06644238846be919893eeecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989f320b06644238846be919893eeecc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40039 } } }
I20260807 18:39:58.353574 24654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "989f320b06644238846be919893eeecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989f320b06644238846be919893eeecc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40039 } } }
I20260807 18:39:58.353852 24654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [sys.catalog]: This master's current role is: LEADER
I20260807 18:39:58.353763 24653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc [sys.catalog]: This master's current role is: LEADER
I20260807 18:39:58.354264 24666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:39:58.357545 24666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:39:58.358619 24647 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:39:58.372803 24678 mvcc.cc:205] Tried to move back new op lower bound from 7315980281293402112 to 7315980281252159488. Current Snapshot: MvccSnapshot[applied={T|T < 7315980281293402112}]
I20260807 18:39:58.377900 24666 catalog_manager.cc:1347] Generated new cluster ID: 848413464ef1416faa8b329b6b8cd423
I20260807 18:39:58.377978 24666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:39:58.399075 24666 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:39:58.400523 24666 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:39:58.418819 24666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc: Generated new TSK 0
I20260807 18:39:58.419821 24666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:39:59.257718 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:60815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40039
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:39:59.416891 24721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:59.417184 24721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:59.417255 24721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:59.421703 24721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:59.421800 24721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:59.421900 24721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:39:59.427522 24721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.138.84:60815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:59.428818 24721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:59.429910 24721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:59.440479 24727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:59.441159 24730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:59.440521 24726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:59.443482 24721 server_base.cc:1034] running on GCE node
I20260807 18:39:59.444103 24721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:59.445050 24721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:59.448365 24721 hybrid_clock.cc:648] HybridClock initialized: now 1786127999448339 us; error 43 us; skew 500 ppm
Aug 07 18:39:59 dist-test-slave-2d19 krb5kdc[23814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:39:59.452694 24721 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:39:59.453681 24721 webserver.cc:460] Webserver started at http://127.6.138.65:42635/ using document root <none> and password file <none>
I20260807 18:39:59.454469 24721 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:59.454579 24721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:59.454825 24721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:39:59 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:39:59.476653 24721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "ade0f54756de4c9a8fa13ed3e679a4ff"
format_stamp: "Formatted at 2026-08-07 18:39:59 on dist-test-slave-2d19"
server_key: "236902098bcb97cae9284d21543e3000"
server_key_iv: "3bf5d204443dbeb107c0cf55892784d8"
server_key_version: "kuduclusterkey@0"
I20260807 18:39:59.477555 24721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "ade0f54756de4c9a8fa13ed3e679a4ff"
format_stamp: "Formatted at 2026-08-07 18:39:59 on dist-test-slave-2d19"
server_key: "236902098bcb97cae9284d21543e3000"
server_key_iv: "3bf5d204443dbeb107c0cf55892784d8"
server_key_version: "kuduclusterkey@0"
I20260807 18:39:59.483314 24721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:39:59.503288 24744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:59.505400 24721 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:39:59.505556 24721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "ade0f54756de4c9a8fa13ed3e679a4ff"
format_stamp: "Formatted at 2026-08-07 18:39:59 on dist-test-slave-2d19"
server_key: "236902098bcb97cae9284d21543e3000"
server_key_iv: "3bf5d204443dbeb107c0cf55892784d8"
server_key_version: "kuduclusterkey@0"
I20260807 18:39:59.505717 24721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:59.540097 24721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:59.543998 24721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:59.544209 24721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:59.545075 24721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:59.546296 24721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:59.546360 24721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:59.546438 24721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:59.546469 24721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:59.567102 24721 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:34177
I20260807 18:39:59.568454 24721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:39:59.569830 24919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:34177 every 8 connection(s)
I20260807 18:39:59.578626 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 24721
I20260807 18:39:59.578764 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:39:59.579128 6697 external_mini_cluster.cc:1428] Setting key 09432823a1e1bde0c302670b7e141a2a
Aug 07 18:39:59 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:39:59.591852 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:59.573980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:49965 (local address 127.6.138.126:40039)
0807 18:39:59.575690 (+ 1710us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:59.575695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:59.577236 (+ 1541us) server_negotiation.cc:407] Connection header received
0807 18:39:59.577293 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:59.577296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:59.577351 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:59.577438 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:59.579282 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:59.580137 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:59.581874 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:59.582236 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:59.582329 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:59.587201 (+ 4872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:59.587205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:59.587208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:39:59.587237 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:59.589258 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:59.589964 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:59.589968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:59.589970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:59.590060 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:59.590467 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:59.590471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:59.590473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:59.590693 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:59.590775 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:39:59.591524 (+ 749us) server_negotiation.cc:299] Negotiation successful
0807 18:39:59.591664 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1623,"thread_start_us":101,"threads_started":1}
I20260807 18:39:59.593772 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:59.572716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:40039 (local address 127.6.138.65:49965)
0807 18:39:59.574446 (+ 1730us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:59.574508 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:59.576912 (+ 2404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:59.577775 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:59.577785 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:59.578154 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:59.579027 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:59.579051 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:59.580318 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:59.580322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:59.581727 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:59.581739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:59.582546 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:59.582550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:59.582763 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:59.583476 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:59.583497 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:59.586983 (+ 3486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:59.589420 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:59.589426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:59.589446 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:59.589820 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:59.590170 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:59.590173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:59.590175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:59.590354 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:59.590845 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:59.590852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:39:59.591191 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 18:39:59.592392 (+ 1201us) client_negotiation.cc:240] Negotiation successful
0807 18:39:59.592733 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1492,"thread_start_us":142,"threads_started":1}
I20260807 18:39:59.595773 24925 heartbeater.cc:347] Connected to a master server at 127.6.138.126:40039
I20260807 18:39:59.596053 24925 heartbeater.cc:464] Registering TS with master...
I20260807 18:39:59.596870 24925 heartbeater.cc:511] Master 127.6.138.126:40039 requested a full tablet report, sending...
I20260807 18:39:59.598821 24589 ts_manager.cc:194] Registered new tserver with Master: ade0f54756de4c9a8fa13ed3e679a4ff (127.6.138.65:34177)
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:39:59.600356 24589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:49965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:39:59.623845 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:60815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40039
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:39:59 dist-test-slave-2d19 krb5kdc[23814](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:39:59.796023 24931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:39:59.796449 24931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:39:59.796520 24931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:39:59.802735 24931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:39:59.802827 24931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:39:59.802925 24931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:39:59.811162 24931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.138.84:60815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:39:59.812768 24931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:39:59.814227 24931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:39:59.827435 24941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:59.827720 24943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:39:59.827493 24940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:39:59.828943 24931 server_base.cc:1034] running on GCE node
I20260807 18:39:59.829455 24931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:39:59.830297 24931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:39:59.831511 24931 hybrid_clock.cc:648] HybridClock initialized: now 1786127999831482 us; error 41 us; skew 500 ppm
Aug 07 18:39:59 dist-test-slave-2d19 krb5kdc[23814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:39:59.835493 24931 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:39:59.836675 24931 webserver.cc:460] Webserver started at http://127.6.138.66:41431/ using document root <none> and password file <none>
I20260807 18:39:59.837437 24931 fs_manager.cc:359] Metadata directory not provided
I20260807 18:39:59.837527 24931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:39:59.837774 24931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:39:59 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:39:59.859730 24931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "9736bafee3374bfea952294383d1149f"
format_stamp: "Formatted at 2026-08-07 18:39:59 on dist-test-slave-2d19"
server_key: "9385fc9e697c064d01abf32d586dd090"
server_key_iv: "07036d12fc4a1c8958cfa498bf7ab8d2"
server_key_version: "kuduclusterkey@0"
I20260807 18:39:59.860602 24931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "9736bafee3374bfea952294383d1149f"
format_stamp: "Formatted at 2026-08-07 18:39:59 on dist-test-slave-2d19"
server_key: "9385fc9e697c064d01abf32d586dd090"
server_key_iv: "07036d12fc4a1c8958cfa498bf7ab8d2"
server_key_version: "kuduclusterkey@0"
I20260807 18:39:59.869724 24931 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260807 18:39:59.907238 24957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:59.912443 24931 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.003s sys 0.000s
I20260807 18:39:59.912618 24931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "9736bafee3374bfea952294383d1149f"
format_stamp: "Formatted at 2026-08-07 18:39:59 on dist-test-slave-2d19"
server_key: "9385fc9e697c064d01abf32d586dd090"
server_key_iv: "07036d12fc4a1c8958cfa498bf7ab8d2"
server_key_version: "kuduclusterkey@0"
I20260807 18:39:59.912781 24931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:39:59.951453 24931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:39:59.955470 24931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:39:59.955680 24931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:39:59.956439 24931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:39:59.957532 24931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:39:59.957598 24931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:59.957652 24931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:39:59.957682 24931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:39:59.969851 24931 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:34797
I20260807 18:39:59.969856 25136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:34797 every 8 connection(s)
I20260807 18:39:59.971475 24931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:39:59.979230 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 24931
I20260807 18:39:59.979367 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:39:59.979677 6697 external_mini_cluster.cc:1428] Setting key b9afd6b443562c672b81d9077247faba
Aug 07 18:39:59 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:00.001971 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:59.974860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:40039 (local address 127.6.138.66:42371)
0807 18:39:59.975531 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 18:39:59.975586 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:39:59.976541 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:39:59.977163 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:39:59.977171 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:39:59.977451 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:39:59.977962 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:59.977976 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:59.978785 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:59.978794 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:59.980143 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:39:59.980153 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:39:59.980803 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:39:59.980807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:39:59.981057 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:59.993344 (+ 12287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:39:59.993372 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:39:59.995311 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:39:59.998007 (+ 2696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:59.998015 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:59.998037 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:59.998437 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:59.998841 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:39:59.998845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:39:59.998847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:39:59.999110 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:39:59.999681 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:39:59.999689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:00.000049 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 18:40:00.000348 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 18:40:00.000753 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":212,"threads_started":1}
I20260807 18:40:00.002913 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:39:59.974054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:42371 (local address 127.6.138.126:40039)
0807 18:39:59.974221 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:39:59.974224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:39:59.975714 (+ 1490us) server_negotiation.cc:407] Connection header received
0807 18:39:59.976804 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:39:59.976807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:39:59.976869 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:39:59.976977 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:39:59.978123 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:59.978653 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:59.980329 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:39:59.980629 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:39:59.980735 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:39:59.995623 (+ 14888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:39:59.995629 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:39:59.995632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:39:59.995665 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:39:59.997825 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:59.998574 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:59.998579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:59.998581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:59.998698 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:39:59.999245 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:39:59.999249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:39:59.999250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:39:59.999512 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:39:59.999615 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:00.002544 (+ 2929us) server_negotiation.cc:299] Negotiation successful
0807 18:40:00.002752 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:40:00.004146 25138 heartbeater.cc:347] Connected to a master server at 127.6.138.126:40039
I20260807 18:40:00.004503 25138 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:00.005172 25138 heartbeater.cc:511] Master 127.6.138.126:40039 requested a full tablet report, sending...
I20260807 18:40:00.006506 24589 ts_manager.cc:194] Registered new tserver with Master: 9736bafee3374bfea952294383d1149f (127.6.138.66:34797)
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:40:00.007292 24589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:42371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:40:00.035933 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:60815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40039
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:40:00 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
W20260807 18:40:00.219913 25144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:00.220289 25144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:00.220353 25144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:00.226187 25144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:00.226259 25144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:00.226342 25144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:40:00.237157 25144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.138.84:60815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:00.238673 25144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:00.240067 25144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:00.254483 25162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:00.254499 25160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:00.255020 25144 server_base.cc:1034] running on GCE node
W20260807 18:40:00.255251 25152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:00.255831 25144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:00.256652 25144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:00.264106 25144 hybrid_clock.cc:648] HybridClock initialized: now 1786128000263920 us; error 192 us; skew 500 ppm
Aug 07 18:40:00 dist-test-slave-2d19 krb5kdc[23814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128000, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:00.268589 25144 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:40:00.269945 25144 webserver.cc:460] Webserver started at http://127.6.138.67:41649/ using document root <none> and password file <none>
I20260807 18:40:00.270726 25144 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:00.270869 25144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:40:00.271173 25144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:40:00 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128000, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:00.294498 25144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9"
format_stamp: "Formatted at 2026-08-07 18:40:00 on dist-test-slave-2d19"
server_key: "c79500d006116e2042d53d104eefca30"
server_key_iv: "e066509fa634b8b939dae22fd0c17ddf"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:00.295434 25144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9"
format_stamp: "Formatted at 2026-08-07 18:40:00 on dist-test-slave-2d19"
server_key: "c79500d006116e2042d53d104eefca30"
server_key_iv: "e066509fa634b8b939dae22fd0c17ddf"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:00.301355 25144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:40:00.324651 25172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:00.326514 25144 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260807 18:40:00.326653 25144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9"
format_stamp: "Formatted at 2026-08-07 18:40:00 on dist-test-slave-2d19"
server_key: "c79500d006116e2042d53d104eefca30"
server_key_iv: "e066509fa634b8b939dae22fd0c17ddf"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:00.326798 25144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:00.346685 25144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:00.350728 25144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:00.351023 25144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:00.351832 25144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:40:00.353130 25144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:40:00.353247 25144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:00.353358 25144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:40:00.353437 25144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:00.371116 25144 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:34465
I20260807 18:40:00.372480 25144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:40:00.374300 25360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:34465 every 8 connection(s)
I20260807 18:40:00.380790 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 25144
I20260807 18:40:00.380927 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:40:00.381246 6697 external_mini_cluster.cc:1428] Setting key edbf2afa2c3b440a68ff173a64c5e01a
Aug 07 18:40:00 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128000, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:00.393635 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.376332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:35157 (local address 127.6.138.126:40039)
0807 18:40:00.376477 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:00.376482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:00.377707 (+ 1225us) server_negotiation.cc:407] Connection header received
0807 18:40:00.379091 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:00.379095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:00.379155 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:00.379262 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:00.382194 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.382989 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.385043 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.385391 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.385461 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.389062 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:00.389067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:00.389070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:00.389097 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:00.390842 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:00.391579 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:00.391582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:00.391584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:00.391676 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:00.392203 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:00.392206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:00.392208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:00.392451 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:00.392545 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:00.393331 (+ 786us) server_negotiation.cc:299] Negotiation successful
0807 18:40:00.393472 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:40:00.395426 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.376853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:40039 (local address 127.6.138.67:35157)
0807 18:40:00.377523 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:00.377587 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:00.378732 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:00.379585 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:00.379597 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:00.380040 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:00.381975 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.381998 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.383168 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.383172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.384905 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.384917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.385665 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.385669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.385907 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.386858 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:00.386879 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:00.388852 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:00.391006 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:00.391014 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:00.391033 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:00.391436 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:00.391805 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:00.391809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:00.391813 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:00.392067 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:00.392582 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:00.392593 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:00.392999 (+ 406us) client_negotiation.cc:769] Sending connection context
0807 18:40:00.393250 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 18:40:00.393675 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":155,"threads_started":1}
I20260807 18:40:00.396929 25361 heartbeater.cc:347] Connected to a master server at 127.6.138.126:40039
I20260807 18:40:00.397217 25361 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:00.397894 25361 heartbeater.cc:511] Master 127.6.138.126:40039 requested a full tablet report, sending...
I20260807 18:40:00.399171 24589 ts_manager.cc:194] Registered new tserver with Master: c4cf0ebd2a9b481c82e46a0675dc0fa9 (127.6.138.67:34465)
I20260807 18:40:00.399920 24589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:35157
I20260807 18:40:00.409253 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:40:00.422322 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.411751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55834 (local address 127.6.138.126:40039)
0807 18:40:00.411919 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:00.411924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:00.412127 (+ 203us) server_negotiation.cc:407] Connection header received
0807 18:40:00.412280 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:00.412284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:00.412341 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:00.412444 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:00.413335 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.414121 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.415312 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.415627 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.415704 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.417616 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:00.417620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:00.417623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:00.417650 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:00.419472 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:00.420671 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:00.420675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:00.420677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:00.420800 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:00.421198 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:00.421202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:00.421204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:00.421467 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:00.421593 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:00.422009 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 18:40:00.422151 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:40:00.430102 24589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:40:00.435678 24589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:40:00.461591 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.454725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:34797 (local address 127.0.0.1:55314)
0807 18:40:00.455132 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:00.455181 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:00.455312 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:00.456141 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:00.456148 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:00.456172 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:00.456371 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.456380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.457999 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.458003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.459537 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.459547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.461065 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.461069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.461175 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.461213 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:40:00.461321 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:40:00.461433 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":87,"threads_started":1}
I20260807 18:40:00.462577 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.454901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.6.138.66:34797)
0807 18:40:00.455596 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:00.455614 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:00.455638 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:40:00.455723 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:00.455728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:00.455947 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:00.456542 (+ 595us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:00.456688 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.457856 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.459708 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.460735 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.460838 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.460989 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:00.462255 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0807 18:40:00.462370 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":156,"threads_started":1}
I20260807 18:40:00.466070 25039 tablet_service.cc:1467] Processing CreateTablet for tablet 532255bc74e34105b8c50bcd34b9ea70 (DEFAULT_TABLE table=test-table [id=60cda636dab34c66b7b432e0b71ce8d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:40:00.467334 25039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 532255bc74e34105b8c50bcd34b9ea70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:00.468295 25390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.456391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34177 (local address 127.0.0.1:58210)
0807 18:40:00.458231 (+ 1840us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:00.458246 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:00.458329 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:00.460175 (+ 1846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:00.460178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:00.460195 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:00.460315 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.460321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.463696 (+ 3375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.463714 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.464815 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.464826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.467884 (+ 3058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.467888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.467988 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.468014 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:40:00.468067 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 18:40:00.468135 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1775,"thread_start_us":103,"threads_started":1}
I20260807 18:40:00.469027 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.456587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58210 (local address 127.6.138.65:34177)
0807 18:40:00.458896 (+ 2309us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:00.458903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:00.459687 (+ 784us) server_negotiation.cc:407] Connection header received
0807 18:40:00.459772 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:00.459778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:00.460021 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:00.460465 (+ 444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:00.460637 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.461670 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.466878 (+ 5208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.467743 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.468571 (+ 828us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.468717 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:00.468769 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 18:40:00.468858 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2206,"thread_start_us":114,"threads_started":1}
I20260807 18:40:00.472014 24816 tablet_service.cc:1467] Processing CreateTablet for tablet 532255bc74e34105b8c50bcd34b9ea70 (DEFAULT_TABLE table=test-table [id=60cda636dab34c66b7b432e0b71ce8d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:40:00.474151 24816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 532255bc74e34105b8c50bcd34b9ea70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:00.478590 25394 tablet_bootstrap.cc:492] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: Bootstrap starting.
I20260807 18:40:00.486008 25394 tablet_bootstrap.cc:654] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:00.487047 25394 log.cc:824] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:00.487782 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.457389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:34465 (local address 127.0.0.1:56850)
0807 18:40:00.458575 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:00.458587 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:00.458665 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:00.463408 (+ 4743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:00.463411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:00.463428 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:00.463548 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.463554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.483315 (+ 19761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.483319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.484841 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.484855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.487325 (+ 2470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.487328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.487440 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.487476 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:40:00.487571 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:40:00.487669 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1117,"thread_start_us":98,"threads_started":1}
I20260807 18:40:00.488415 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.457798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.6.138.67:34465)
0807 18:40:00.461952 (+ 4154us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:00.461966 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:00.461990 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:40:00.462069 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:00.462076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:00.463114 (+ 1038us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:00.463265 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:00.464949 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.466032 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.485038 (+ 19006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.486959 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.487080 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.487253 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:00.488104 (+ 851us) server_negotiation.cc:299] Negotiation successful
0807 18:40:00.488204 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4044,"thread_start_us":111,"threads_started":1}
I20260807 18:40:00.488875 25395 tablet_bootstrap.cc:492] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff: Bootstrap starting.
Aug 07 18:40:00 dist-test-slave-2d19 krb5kdc[23814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:00.491452 25395 tablet_bootstrap.cc:654] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:00.491616 25249 tablet_service.cc:1467] Processing CreateTablet for tablet 532255bc74e34105b8c50bcd34b9ea70 (DEFAULT_TABLE table=test-table [id=60cda636dab34c66b7b432e0b71ce8d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:40:00.492403 25395 log.cc:824] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:00.492852 25249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 532255bc74e34105b8c50bcd34b9ea70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:00.494706 25394 tablet_bootstrap.cc:492] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: No bootstrap required, opened a new log
I20260807 18:40:00.494933 25394 ts_tablet_manager.cc:1397] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20260807 18:40:00.499143 25394 raft_consensus.cc:348] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } }
I20260807 18:40:00.499441 25394 raft_consensus.cc:374] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:00.499500 25394 raft_consensus.cc:729] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9736bafee3374bfea952294383d1149f, State: Initialized, Role: FOLLOWER
I20260807 18:40:00.499979 25394 consensus_queue.cc:260] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } }
I20260807 18:40:00.501096 25394 ts_tablet_manager.cc:1428] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 18:40:00.501477 25138 heartbeater.cc:503] Master 127.6.138.126:40039 was elected leader, sending a full tablet report...
I20260807 18:40:00.514035 25404 tablet_bootstrap.cc:492] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: Bootstrap starting.
I20260807 18:40:00.520756 25400 raft_consensus.cc:482] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:40:00.521019 25400 raft_consensus.cc:504] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } }
I20260807 18:40:00.524048 25404 tablet_bootstrap.cc:654] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:00.526819 25395 tablet_bootstrap.cc:492] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff: No bootstrap required, opened a new log
I20260807 18:40:00.527094 25395 ts_tablet_manager.cc:1397] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.004s
I20260807 18:40:00.532385 25395 raft_consensus.cc:348] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } }
I20260807 18:40:00.532771 25395 raft_consensus.cc:374] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:00.532837 25395 raft_consensus.cc:729] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade0f54756de4c9a8fa13ed3e679a4ff, State: Initialized, Role: FOLLOWER
I20260807 18:40:00.533555 25395 consensus_queue.cc:260] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } }
I20260807 18:40:00.534468 25400 leader_election.cc:288] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ade0f54756de4c9a8fa13ed3e679a4ff (127.6.138.65:34177), c4cf0ebd2a9b481c82e46a0675dc0fa9 (127.6.138.67:34465)
I20260807 18:40:00.535213 25395 ts_tablet_manager.cc:1428] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260807 18:40:00.535638 24925 heartbeater.cc:503] Master 127.6.138.126:40039 was elected leader, sending a full tablet report...
I20260807 18:40:00.535952 25404 log.cc:824] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:00.547585 25404 tablet_bootstrap.cc:492] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: No bootstrap required, opened a new log
I20260807 18:40:00.547952 25404 ts_tablet_manager.cc:1397] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: Time spent bootstrapping tablet: real 0.045s user 0.008s sys 0.000s
I20260807 18:40:00.549453 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.531158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:41183 (local address 127.6.138.65:34177)
0807 18:40:00.531329 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:00.531334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:00.539876 (+ 8542us) server_negotiation.cc:407] Connection header received
0807 18:40:00.540108 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:00.540111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:00.540161 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:00.540208 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:00.540687 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.541411 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.544032 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.544616 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.544732 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.544801 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:00.549177 (+ 4376us) server_negotiation.cc:299] Negotiation successful
0807 18:40:00.549302 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:40:00.549952 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.531027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34177 (local address 127.6.138.66:41183)
0807 18:40:00.539775 (+ 8748us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:00.539793 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:00.539975 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:00.540392 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:00.540395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:00.540415 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:00.540544 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.540551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.541576 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.541580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.542763 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.542776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.544914 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.544918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.545029 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.545055 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:40:00.549679 (+ 4624us) client_negotiation.cc:240] Negotiation successful
0807 18:40:00.549774 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8663,"thread_start_us":169,"threads_started":1}
I20260807 18:40:00.550659 24852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532255bc74e34105b8c50bcd34b9ea70" candidate_uuid: "9736bafee3374bfea952294383d1149f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" is_pre_election: true
I20260807 18:40:00.551007 24852 raft_consensus.cc:2454] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9736bafee3374bfea952294383d1149f in term 0.
I20260807 18:40:00.551592 24959 leader_election.cc:302] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9736bafee3374bfea952294383d1149f, ade0f54756de4c9a8fa13ed3e679a4ff; no voters:
I20260807 18:40:00.552027 25405 raft_consensus.cc:2781] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:40:00.552145 25405 raft_consensus.cc:482] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:40:00.552211 25405 raft_consensus.cc:3037] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:00.553825 25405 raft_consensus.cc:504] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } }
I20260807 18:40:00.554623 24852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532255bc74e34105b8c50bcd34b9ea70" candidate_uuid: "9736bafee3374bfea952294383d1149f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff"
I20260807 18:40:00.554773 24852 raft_consensus.cc:3037] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:00.556212 24852 raft_consensus.cc:2454] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9736bafee3374bfea952294383d1149f in term 1.
I20260807 18:40:00.556598 24959 leader_election.cc:302] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9736bafee3374bfea952294383d1149f, ade0f54756de4c9a8fa13ed3e679a4ff; no voters:
I20260807 18:40:00.556792 25400 raft_consensus.cc:2781] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:40:00.557075 25400 raft_consensus.cc:686] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 1 LEADER]: Becoming Leader. State: Replica: 9736bafee3374bfea952294383d1149f, State: Running, Role: LEADER
I20260807 18:40:00.557492 25400 consensus_queue.cc:237] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } }
I20260807 18:40:00.558153 25404 raft_consensus.cc:348] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } }
I20260807 18:40:00.558840 25404 raft_consensus.cc:374] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:00.558954 25404 raft_consensus.cc:729] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4cf0ebd2a9b481c82e46a0675dc0fa9, State: Initialized, Role: FOLLOWER
I20260807 18:40:00.559811 25404 consensus_queue.cc:260] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } }
I20260807 18:40:00.561525 25404 ts_tablet_manager.cc:1428] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260807 18:40:00.562279 25361 heartbeater.cc:503] Master 127.6.138.126:40039 was elected leader, sending a full tablet report...
I20260807 18:40:00.562453 24589 catalog_manager.cc:5495] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9736bafee3374bfea952294383d1149f (127.6.138.66). New cstate: current_term: 1 leader_uuid: "9736bafee3374bfea952294383d1149f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9736bafee3374bfea952294383d1149f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 } health_report { overall_health: UNKNOWN } } }
I20260807 18:40:00.564096 25405 leader_election.cc:288] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [CANDIDATE]: Term 1 election: Requested vote from peers ade0f54756de4c9a8fa13ed3e679a4ff (127.6.138.65:34177), c4cf0ebd2a9b481c82e46a0675dc0fa9 (127.6.138.67:34465)
I20260807 18:40:00.567411 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.534282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:46757 (local address 127.6.138.67:34465)
0807 18:40:00.534413 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:00.534417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:00.543119 (+ 8702us) server_negotiation.cc:407] Connection header received
0807 18:40:00.543357 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:00.543361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:00.543448 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:00.543517 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:00.545200 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.545884 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.564366 (+ 18482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.565023 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.565135 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.565191 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:00.567129 (+ 1938us) server_negotiation.cc:299] Negotiation successful
0807 18:40:00.567231 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:40:00.568019 25411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.534217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:34465 (local address 127.6.138.66:46757)
0807 18:40:00.543025 (+ 8808us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:00.543040 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:00.543212 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:00.543680 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:00.543683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:00.543703 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:00.543851 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.543860 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.547829 (+ 3969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.547831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.549013 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:00.549025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.565312 (+ 16287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.565316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:00.566912 (+ 1596us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.566946 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:40:00.567694 (+ 748us) client_negotiation.cc:240] Negotiation successful
0807 18:40:00.567848 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8736,"thread_start_us":104,"threads_started":1}
I20260807 18:40:00.568800 25278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532255bc74e34105b8c50bcd34b9ea70" candidate_uuid: "9736bafee3374bfea952294383d1149f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" is_pre_election: true
I20260807 18:40:00.569137 25278 raft_consensus.cc:2454] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9736bafee3374bfea952294383d1149f in term 0.
I20260807 18:40:00.569519 25277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "532255bc74e34105b8c50bcd34b9ea70" candidate_uuid: "9736bafee3374bfea952294383d1149f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9"
I20260807 18:40:00.569640 25277 raft_consensus.cc:3037] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:00.571070 25277 raft_consensus.cc:2454] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9736bafee3374bfea952294383d1149f in term 1.
W20260807 18:40:00.580513 24926 tablet.cc:2369] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:40:00.603931 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.598238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.6.138.66:34797)
0807 18:40:00.598501 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:00.598507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:00.598523 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:40:00.598616 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:00.598621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:00.598697 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:00.598800 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:40:00.599231 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.599989 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.601497 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:00.601845 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:00.601976 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:00.602464 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:40:00.603096 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:40:00.603229 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:00.603707 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 18:40:00.603773 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260807 18:40:00.615818 25277 raft_consensus.cc:1261] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [term 1 FOLLOWER]: Refusing update from remote peer 9736bafee3374bfea952294383d1149f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:40:00.616078 24852 raft_consensus.cc:1261] T 532255bc74e34105b8c50bcd34b9ea70 P ade0f54756de4c9a8fa13ed3e679a4ff [term 1 FOLLOWER]: Refusing update from remote peer 9736bafee3374bfea952294383d1149f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:40:00.617230 25405 consensus_queue.cc:1035] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ade0f54756de4c9a8fa13ed3e679a4ff" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:40:00.617731 25405 consensus_queue.cc:1035] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4cf0ebd2a9b481c82e46a0675dc0fa9" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 34465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:40:00.624914 25363 tablet.cc:2369] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:40:00.642874 25425 mvcc.cc:205] Tried to move back new op lower bound from 7315980290499768320 to 7315980290286936064. Current Snapshot: MvccSnapshot[applied={T|T < 7315980290499768320}]
I20260807 18:40:00.643325 25426 mvcc.cc:205] Tried to move back new op lower bound from 7315980290499768320 to 7315980290286936064. Current Snapshot: MvccSnapshot[applied={T|T < 7315980290499768320}]
I20260807 18:40:00.652812 25428 mvcc.cc:205] Tried to move back new op lower bound from 7315980290499768320 to 7315980290286936064. Current Snapshot: MvccSnapshot[applied={T|T < 7315980290499768320}]
I20260807 18:40:00.671561 24587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:40:00.671775 24587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:40:00.675277 24587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 989f320b06644238846be919893eeecc: Sending DeleteTablet for 3 replicas of tablet 532255bc74e34105b8c50bcd34b9ea70
I20260807 18:40:00.676685 25249 tablet_service.cc:1514] Processing DeleteTablet for tablet 532255bc74e34105b8c50bcd34b9ea70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:40:00 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:56850
I20260807 18:40:00.676868 25039 tablet_service.cc:1514] Processing DeleteTablet for tablet 532255bc74e34105b8c50bcd34b9ea70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:40:00 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:55314
I20260807 18:40:00.678258 24816 tablet_service.cc:1514] Processing DeleteTablet for tablet 532255bc74e34105b8c50bcd34b9ea70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:40:00 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:58210
I20260807 18:40:00.681842 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 24721
I20260807 18:40:00.695243 25441 tablet_replica.cc:330] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: stopping tablet replica
I20260807 18:40:00.695779 25441 raft_consensus.cc:2229] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:40:00.696444 25441 raft_consensus.cc:2258] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:40:00.711596 25441 ts_tablet_manager.cc:1905] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:40:00.715525 25441 ts_tablet_manager.cc:1918] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:40:00.715662 25441 log.cc:1196] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/532255bc74e34105b8c50bcd34b9ea70
I20260807 18:40:00.716027 25441 ts_tablet_manager.cc:1939] T 532255bc74e34105b8c50bcd34b9ea70 P 9736bafee3374bfea952294383d1149f: Deleting consensus metadata
I20260807 18:40:00.716503 25440 tablet_replica.cc:330] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: stopping tablet replica
I20260807 18:40:00.716922 25440 raft_consensus.cc:2229] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:40:00.717377 25440 raft_consensus.cc:2258] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 18:40:00.724330 24569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:34177 (error 108)
I20260807 18:40:00.725369 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 24931
I20260807 18:40:00.725521 25440 ts_tablet_manager.cc:1905] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:40:00.728766 25440 ts_tablet_manager.cc:1918] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:40:00.728860 25440 log.cc:1196] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/532255bc74e34105b8c50bcd34b9ea70
I20260807 18:40:00.729511 25440 ts_tablet_manager.cc:1939] T 532255bc74e34105b8c50bcd34b9ea70 P c4cf0ebd2a9b481c82e46a0675dc0fa9: Deleting consensus metadata
I20260807 18:40:00.734241 24568 catalog_manager.cc:4841] TS c4cf0ebd2a9b481c82e46a0675dc0fa9 (127.6.138.67:34465): tablet 532255bc74e34105b8c50bcd34b9ea70 (table test-table [id=60cda636dab34c66b7b432e0b71ce8d3]) successfully deleted
I20260807 18:40:00.736888 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:00.736235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34177 (local address 127.0.0.1:58214)
0807 18:40:00.736698 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:00.736790 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:34177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366}
W20260807 18:40:00.737251 24569 catalog_manager.cc:4584] TS ade0f54756de4c9a8fa13ed3e679a4ff (127.6.138.65:34177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 532255bc74e34105b8c50bcd34b9ea70: Network error: Client connection negotiation failed: client connection to 127.6.138.65:34177: connect: Connection refused (error 111)
W20260807 18:40:00.747218 25392 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:40:00.746566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:34797 (local address 127.0.0.1:55326)
0807 18:40:00.746764 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:00.746781 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:00.746880 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:00.747089 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.66:34797: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":115}
I20260807 18:40:00.748001 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 25144
I20260807 18:40:00.762458 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 24536
I20260807 18:40:00.780731 6697 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:40:01.136653 6697 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:40:01.136813 6697 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:40:01.493413 6697 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:40:01.493 UTC [23867] LOG: received smart shutdown request
2026-08-07 18:40:01.498 UTC [23867] LOG: background worker "logical replication launcher" (PID 23881) exited with exit code 1
2026-08-07 18:40:01.498 UTC [23871] LOG: shutting down
2026-08-07 18:40:01.675 UTC [23867] LOG: database system is shut down
2026-08-07T18:40:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50574 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:40:01 dist-test-slave-2d19 krb5kdc[25456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:40:01 dist-test-slave-2d19 krb5kdc[25456](info): set up 2 sockets
Aug 07 18:40:01 dist-test-slave-2d19 krb5kdc[25456](info): commencing operation
krb5kdc: starting...
W20260807 18:40:03.080315 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:40:03 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:40:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:40:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.138.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.138.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/rangeradmin_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/rangeradmin_127.6.138.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.138.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.138.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/rangerlookup_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/rangerlookup_127.6.138.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.138.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.138.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/HTTP_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/HTTP_127.6.138.84@KRBTEST.COM.keytab.
I20260807 18:40:03.270741 6697 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskPYlYkN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/postgres -l logfile start
2026-08-07 18:40:05.887 UTC [25518] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:40:05.887 UTC [25518] LOG: listening on IPv4 address "127.6.138.84", port 45327
2026-08-07 18:40:05.891 UTC [25518] LOG: listening on Unix socket "/tmp/.s.PGSQL.45327"
2026-08-07 18:40:05.913 UTC [25522] LOG: database system was shut down at 2026-08-07 18:40:04 UTC
2026-08-07 18:40:05.917 UTC [25518] LOG: database system is ready to accept connections
I20260807 18:40:07.098497 6697 mini_postgres.cc:96] Postgres bound to 45327
2026-08-07 18:40:07.106 UTC [25537] FATAL: database "slave" does not exist
127.6.138.84:45327 - accepting connections
I20260807 18:40:07.106604 6697 mini_ranger.cc:161] Starting Ranger...
I20260807 18:40:07.122860 6697 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:40:07.628116 6697 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:40:07.628209 6697 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-admin
2026-08-07 18:40:07,922 [I] DB FLAVOR :POSTGRES
2026-08-07 18:40:07,922 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:40:07,922 [I] Checking connection..
2026-08-07 18:40:07,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:40:08,173 [I] Checking connection passed.
2026-08-07 18:40:08,174 [I] --------- Verifying version history table ---------
2026-08-07 18:40:08,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:40:08,431 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:40:08,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:40:08,688 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:40:08,688 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:40:08,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:40:08.928 UTC [25629] WARNING: there is no transaction in progress
2026-08-07 18:40:08,937 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:40:08,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:40:09,199 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:40:09,199 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:40:09,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:40:09,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:40:09,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2d19.c.gcp-upstream.internal','N') ;"
2026-08-07 18:40:10,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:40:10,282 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:40:10,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:40:10,538 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:40:10,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:40:10,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:40:11,086 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:40:11,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:40:12.044 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.061 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.077 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.092 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.109 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.126 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.190 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.197 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.210 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.220 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.229 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.244 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.254 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.260 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.267 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12.274 UTC [25806] WARNING: there is no transaction in progress
2026-08-07 18:40:12,908 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:40:12,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:40:13,171 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:40:13,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:40:13,441 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:40:13,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:40:13,680 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:40:13,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:40:13,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:40:14,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2d19.c.gcp-upstream.internal';"
2026-08-07 18:40:14,493 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:40:14,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:40:14,759 [I] DB_PATCHES have already been applied
I20260807 18:40:14.765026 6697 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-admin:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskPYlYkN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:40:14.765125 6697 mini_ranger.cc:193] Using host: 127.6.138.84
I20260807 18:40:14.769620 6697 mini_ranger.cc:227] Ranger admin URL: http://127.6.138.84:58269
Aug 07, 2026 6:40:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:40:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:40:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:40:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:40:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:40:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:40:15 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:40:16.114526 6697 mini_ranger.cc:160] Time spent starting Ranger: real 9.008s user 0.006s sys 0.007s
Aug 07, 2026 6:40:36 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:40:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 6:40:36 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:40:37 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:40:38 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:40:38.721993 6697 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.138.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.138.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/rangerkms_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.138.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/rangerkms_127.6.138.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.138.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.138.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.138.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/HTTP_127.6.138.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.138.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/HTTP_127.6.138.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:40:38 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128038, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:40:38.815670 6697 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:40:38.827354 6697 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:40:39.745581 6697 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:40:39.745678 6697 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-kms
I20260807 18:40:39.745697 6697 mini_ranger_kms.cc:227] Using postgres at 127.6.138.84:45327
2026-08-07 18:40:39,938 [I] DB FLAVOR :POSTGRES
2026-08-07 18:40:39,938 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:40:39,938 [I] Checking connection
2026-08-07 18:40:39,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:40:40,180 [I] connection success
2026-08-07 18:40:40,181 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:40:40,181 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:40:40,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:40:40,438 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:40:40,438 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:40:40,438 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:40:40,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.138.84:45327/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:40:40,713 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:40:41.278961 6697 mini_ranger_kms.cc:310] Created kms service
I20260807 18:40:41.512176 6697 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:40:41.559623 6697 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:40:41 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128038, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:41.991653 6697 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:40:41.991816 6697 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-kms:/tmp/dist-test-taskPYlYkN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskPYlYkN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:40:41.991848 6697 mini_ranger_kms.cc:240] Using host: 127.6.138.84
I20260807 18:40:41.998188 6697 mini_ranger_kms.cc:276] Ranger KMS PID: 26096
I20260807 18:40:41.998282 6697 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.138.84:51107
Aug 07, 2026 6:40:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:40:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:40:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:40:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:40:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:40:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 18:40:44 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786128044, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.138.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:44.828845 6697 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.013s user 0.017s sys 0.161s
I20260807 18:40:44.829010 6697 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:40:44.829089 6697 mini_ranger_kms.cc:401] 127.6.138.84:51107/kms/v1/keys
Aug 07 18:40:44 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128044, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.138.84@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
Aug 07 18:40:45 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128044, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.138.84@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
Aug 07 18:40:47 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:40:47.640461 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:46133
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:46133
--ranger_config_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:51107/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:40:47.758541 26194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:47.758816 26194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:47.758855 26194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:47.762876 26194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:40:47.762948 26194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:47.762969 26194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:47.762984 26194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:40:47.763000 26194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:40:47.768724 26194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.138.84:51107/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:46133
--ranger_config_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:46133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:47.769881 26194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:47.770946 26194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:47.781117 26211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:47.781093 26208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:47.781356 26194 server_base.cc:1034] running on GCE node
W20260807 18:40:47.781122 26206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:47.782079 26194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:47.783294 26194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:47.784489 26194 hybrid_clock.cc:648] HybridClock initialized: now 1786128047784472 us; error 35 us; skew 500 ppm
Aug 07 18:40:47 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128047, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:47.787544 26194 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:40:47.788756 26194 webserver.cc:460] Webserver started at http://127.6.138.126:40723/ using document root <none> and password file <none>
I20260807 18:40:47.789462 26194 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:47.789554 26194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:40:47.789819 26194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:40:47 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128047, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:47.856747 26194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "2d2e8dfc8f0e481ebef10a01bea0f779"
format_stamp: "Formatted at 2026-08-07 18:40:47 on dist-test-slave-2d19"
server_key: "1bebc100e7be98bd66d34558b3d8ff7e"
server_key_iv: "d051f3bf0d7ca29fde8df30fcc06bb86"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:47.857532 26194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "2d2e8dfc8f0e481ebef10a01bea0f779"
format_stamp: "Formatted at 2026-08-07 18:40:47 on dist-test-slave-2d19"
server_key: "1bebc100e7be98bd66d34558b3d8ff7e"
server_key_iv: "d051f3bf0d7ca29fde8df30fcc06bb86"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:47.861919 26194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:40:47.880530 26225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:47.881997 26194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:40:47.882153 26194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "2d2e8dfc8f0e481ebef10a01bea0f779"
format_stamp: "Formatted at 2026-08-07 18:40:47 on dist-test-slave-2d19"
server_key: "1bebc100e7be98bd66d34558b3d8ff7e"
server_key_iv: "d051f3bf0d7ca29fde8df30fcc06bb86"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:47.882302 26194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:47.903532 26194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:47.906630 26194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:47.906843 26194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:47.915374 26194 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:46133
I20260807 18:40:47.915383 26319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:46133 every 8 connection(s)
I20260807 18:40:47.916481 26194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:40:47.919557 26320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:47.919899 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 26194
I20260807 18:40:47.920029 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:40:47.920401 6697 external_mini_cluster.cc:1428] Setting key 31c1eb2acd94b2974cf96f7299f2d554
I20260807 18:40:47.926973 26320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Bootstrap starting.
I20260807 18:40:47.929757 26320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:47.930781 26320 log.cc:824] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:40:47 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:47.933274 26320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: No bootstrap required, opened a new log
I20260807 18:40:47.937472 26320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } }
I20260807 18:40:47.937765 26320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:47.937865 26320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2e8dfc8f0e481ebef10a01bea0f779, State: Initialized, Role: FOLLOWER
I20260807 18:40:47.938257 26325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:47.921925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.6.138.126:46133)
0807 18:40:47.922404 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:47.922424 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:47.922469 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:40:47.923410 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:47.923434 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:47.923887 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:47.924379 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:47.925310 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:47.926844 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:47.928023 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:47.928748 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:47.928906 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:47.932040 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:47.932047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:47.932064 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:47.932093 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:47.934829 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:47.935375 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:47.935381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:47.935389 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:47.935530 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:47.935855 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:47.935859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:47.935860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:47.936449 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:47.936682 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:47.936955 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 18:40:47.937242 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":132,"threads_started":1}
I20260807 18:40:47.938395 26320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } }
I20260807 18:40:47.938572 26320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:40:47.938656 26320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:40:47.938772 26320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:47.939913 26320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } }
I20260807 18:40:47.940330 26320 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d2e8dfc8f0e481ebef10a01bea0f779; no voters:
I20260807 18:40:47.940702 26320 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:40:47.941087 26329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:40:47.941277 26329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 1 LEADER]: Becoming Leader. State: Replica: 2d2e8dfc8f0e481ebef10a01bea0f779, State: Running, Role: LEADER
I20260807 18:40:47.941648 26329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } }
I20260807 18:40:47.942067 26320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:40:47.942864 26329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2e8dfc8f0e481ebef10a01bea0f779. Latest consensus state: current_term: 1 leader_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } } }
I20260807 18:40:47.943082 26329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: This master's current role is: LEADER
I20260807 18:40:47.943382 26330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } } }
I20260807 18:40:47.943516 26341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:40:47.943552 26330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: This master's current role is: LEADER
I20260807 18:40:47.948014 26341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:40:47.948495 26320 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:40:47.958777 26341 catalog_manager.cc:1347] Generated new cluster ID: 83fd58670e63432c98fa1d44a5b979dd
I20260807 18:40:47.958874 26341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:40:47.971451 26341 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:40:47.972867 26341 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:40:47.991801 26341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Generated new TSK 0
I20260807 18:40:47.992882 26341 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:40:48.721810 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:51107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:46133
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:40:48.875001 26406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:48.875288 26406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:48.875331 26406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:48.879984 26406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:48.880064 26406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:48.880162 26406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:40:48.887910 26406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.138.84:51107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:46133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:48.889151 26406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:48.890268 26406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:48.901741 26423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:48.901793 26420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:48.902346 26406 server_base.cc:1034] running on GCE node
W20260807 18:40:48.902513 26417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:48.903009 26406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:48.903818 26406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:48.905009 26406 hybrid_clock.cc:648] HybridClock initialized: now 1786128048904984 us; error 40 us; skew 500 ppm
Aug 07 18:40:48 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128048, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:48.908658 26406 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:40:48.909857 26406 webserver.cc:460] Webserver started at http://127.6.138.65:44553/ using document root <none> and password file <none>
I20260807 18:40:48.910643 26406 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:48.910755 26406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:40:48.911026 26406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:40:48 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128048, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:48.931067 26406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46"
format_stamp: "Formatted at 2026-08-07 18:40:48 on dist-test-slave-2d19"
server_key: "e9705ccfe5bf3f23ec550486db600c9d"
server_key_iv: "7232e07150b6a2748301a7990dd37073"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:48.932018 26406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46"
format_stamp: "Formatted at 2026-08-07 18:40:48 on dist-test-slave-2d19"
server_key: "e9705ccfe5bf3f23ec550486db600c9d"
server_key_iv: "7232e07150b6a2748301a7990dd37073"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:48.937417 26406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:40:48.958374 26431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:48.960160 26406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:40:48.960397 26406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46"
format_stamp: "Formatted at 2026-08-07 18:40:48 on dist-test-slave-2d19"
server_key: "e9705ccfe5bf3f23ec550486db600c9d"
server_key_iv: "7232e07150b6a2748301a7990dd37073"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:48.960641 26406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:48.982980 26406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:48.986783 26406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:48.987073 26406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:48.987901 26406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:40:48.989125 26406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:40:48.989248 26406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:48.989383 26406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:40:48.989426 26406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:49.000680 26406 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:39825
I20260807 18:40:49.000852 26621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:39825 every 8 connection(s)
I20260807 18:40:49.001758 26406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:40:49.010849 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 26406
I20260807 18:40:49.011006 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:40:49.011372 6697 external_mini_cluster.cc:1428] Setting key c35a76e5cf951509c67f2eacf14a26b7
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128048, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:49.017403 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:49.003819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:53327 (local address 127.6.138.126:46133)
0807 18:40:49.004087 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:49.004092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:49.005001 (+ 909us) server_negotiation.cc:407] Connection header received
0807 18:40:49.006052 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:49.006055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:49.006128 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:49.006243 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:49.007364 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.007927 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.009156 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.009402 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.009520 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:49.012404 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:49.012408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:49.012410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:49.012434 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:49.014471 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:49.015210 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:49.015216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:49.015221 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:49.015322 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:49.015851 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:49.015856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:49.015858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:49.016122 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:49.016213 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:49.017022 (+ 809us) server_negotiation.cc:299] Negotiation successful
0807 18:40:49.017177 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
I20260807 18:40:49.019588 26627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:49.004359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:46133 (local address 127.6.138.65:53327)
0807 18:40:49.004837 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:49.004890 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:49.005825 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:49.006433 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:49.006441 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:49.006724 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:49.007207 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:49.007222 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.008060 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.008064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:49.009014 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:49.009024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.009587 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.009592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:49.009759 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:49.010399 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:49.010416 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:49.012180 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:49.014648 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:49.014659 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:49.014680 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:49.015074 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:49.015437 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:49.015443 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:49.015448 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:49.015678 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:49.016312 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:49.016322 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:49.016705 (+ 383us) client_negotiation.cc:769] Sending connection context
0807 18:40:49.017965 (+ 1260us) client_negotiation.cc:240] Negotiation successful
0807 18:40:49.018350 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":144,"threads_started":1}
I20260807 18:40:49.021426 26623 heartbeater.cc:347] Connected to a master server at 127.6.138.126:46133
I20260807 18:40:49.021705 26623 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:49.022347 26623 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
I20260807 18:40:49.024402 26261 ts_manager.cc:194] Registered new tserver with Master: 0f248c4d9eaf43e9b0cbdabaedf1fa46 (127.6.138.65:39825)
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:40:49.025709 26261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:53327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:40:49.045441 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:51107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:46133
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:40:49.211459 26633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:49.211913 26633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:49.212039 26633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:49.218364 26633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:49.218451 26633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:49.218549 26633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:40:49.226624 26633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.138.84:51107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:46133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:49.228008 26633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:49.229344 26633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:49.249141 26646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:49.249616 26633 server_base.cc:1034] running on GCE node
W20260807 18:40:49.249833 26644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:49.249971 26643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:49.250602 26633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:49.251483 26633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:49.257264 26633 hybrid_clock.cc:648] HybridClock initialized: now 1786128049257174 us; error 99 us; skew 500 ppm
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:49.261837 26633 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:40:49.263063 26633 webserver.cc:460] Webserver started at http://127.6.138.66:45333/ using document root <none> and password file <none>
I20260807 18:40:49.263895 26633 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:49.264070 26633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:40:49.264428 26633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:49.283022 26633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "af8394accc02427fb11162a43b3401da"
format_stamp: "Formatted at 2026-08-07 18:40:49 on dist-test-slave-2d19"
server_key: "6e21353ffd63c3b0b29824d6e468f904"
server_key_iv: "83dac9837c9bfe76c547759104ab06bd"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:49.283934 26633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "af8394accc02427fb11162a43b3401da"
format_stamp: "Formatted at 2026-08-07 18:40:49 on dist-test-slave-2d19"
server_key: "6e21353ffd63c3b0b29824d6e468f904"
server_key_iv: "83dac9837c9bfe76c547759104ab06bd"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:49.291957 26633 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.002s
I20260807 18:40:49.310874 26655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:49.312767 26633 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:40:49.312906 26633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "af8394accc02427fb11162a43b3401da"
format_stamp: "Formatted at 2026-08-07 18:40:49 on dist-test-slave-2d19"
server_key: "6e21353ffd63c3b0b29824d6e468f904"
server_key_iv: "83dac9837c9bfe76c547759104ab06bd"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:49.313046 26633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:49.348639 26633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:49.351979 26633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:49.352202 26633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:49.352982 26633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:40:49.353991 26633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:40:49.354068 26633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:49.354159 26633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:40:49.354206 26633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:49.388276 26633 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:36721
I20260807 18:40:49.389950 26633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:40:49.398355 26836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:36721 every 8 connection(s)
I20260807 18:40:49.399477 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 26633
I20260807 18:40:49.399633 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:40:49.399948 6697 external_mini_cluster.cc:1428] Setting key 440b1f15d749e99a98b20efcce42d32e
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
I20260807 18:40:49.423611 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:49.396612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:46815 (local address 127.6.138.126:46133)
0807 18:40:49.396833 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:49.396839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:49.399702 (+ 2863us) server_negotiation.cc:407] Connection header received
0807 18:40:49.404408 (+ 4706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:49.404411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:49.404473 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:49.404571 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:49.406357 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.407239 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.409250 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.409702 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.409807 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:49.418655 (+ 8848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:49.418663 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:49.418669 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:49.418711 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:49.420793 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:49.421598 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:49.421602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:49.421604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:49.421704 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:49.422159 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:49.422163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:49.422164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:49.422402 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:49.422485 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:49.423266 (+ 781us) server_negotiation.cc:299] Negotiation successful
0807 18:40:49.423420 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 18:40:49.425729 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:49.396050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:46133 (local address 127.6.138.66:46815)
0807 18:40:49.399478 (+ 3428us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:49.399561 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:49.400883 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:49.404941 (+ 4058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:49.404954 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:49.405378 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:49.406163 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:49.406190 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.407418 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.407423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:49.409078 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:49.409093 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.410076 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.410084 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:49.410394 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:49.413177 (+ 2783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:49.413203 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:49.418338 (+ 5135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:49.420996 (+ 2658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:49.421004 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:49.421029 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:49.421436 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:49.421826 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:49.421830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:49.421832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:49.422037 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:49.422573 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:49.422581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:49.422939 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 18:40:49.424133 (+ 1194us) client_negotiation.cc:240] Negotiation successful
0807 18:40:49.424575 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3148,"thread_start_us":966,"threads_started":1}
I20260807 18:40:49.427449 26837 heartbeater.cc:347] Connected to a master server at 127.6.138.126:46133
I20260807 18:40:49.427765 26837 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:49.428560 26837 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
I20260807 18:40:49.429832 26261 ts_manager.cc:194] Registered new tserver with Master: af8394accc02427fb11162a43b3401da (127.6.138.66:36721)
I20260807 18:40:49.430691 26261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:46815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:40:49.452194 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:51107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:46133
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
W20260807 18:40:49.619093 26847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:49.619418 26847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:49.619525 26847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:49.623915 26847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:49.624013 26847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:49.624130 26847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:40:49.632444 26847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.138.84:51107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:46133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:49.634312 26847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:49.635800 26847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:49.650722 26853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:49.650725 26860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:49.651611 26847 server_base.cc:1034] running on GCE node
W20260807 18:40:49.652395 26856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:49.652923 26847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:49.653847 26847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:49.655174 26847 hybrid_clock.cc:648] HybridClock initialized: now 1786128049655104 us; error 83 us; skew 500 ppm
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:49.658910 26847 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:40:49.660025 26847 webserver.cc:460] Webserver started at http://127.6.138.67:39383/ using document root <none> and password file <none>
I20260807 18:40:49.660785 26847 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:49.660928 26847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:40:49.661191 26847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:49.677400 26847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "2550682d0a8e47e8aea2d45f1ff74caa"
format_stamp: "Formatted at 2026-08-07 18:40:49 on dist-test-slave-2d19"
server_key: "9362badc19ecb7399f025d7d7ffa0961"
server_key_iv: "2ee1a0ec3fe1a340d5ea740c0cdc9621"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:49.678198 26847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "2550682d0a8e47e8aea2d45f1ff74caa"
format_stamp: "Formatted at 2026-08-07 18:40:49 on dist-test-slave-2d19"
server_key: "9362badc19ecb7399f025d7d7ffa0961"
server_key_iv: "2ee1a0ec3fe1a340d5ea740c0cdc9621"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:49.685724 26847 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260807 18:40:49.702119 26872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:49.703778 26847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:40:49.703958 26847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "2550682d0a8e47e8aea2d45f1ff74caa"
format_stamp: "Formatted at 2026-08-07 18:40:49 on dist-test-slave-2d19"
server_key: "9362badc19ecb7399f025d7d7ffa0961"
server_key_iv: "2ee1a0ec3fe1a340d5ea740c0cdc9621"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:49.704107 26847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:49.713842 26847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:49.717213 26847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:49.717413 26847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:49.718086 26847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:40:49.719005 26847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:40:49.719074 26847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:49.719143 26847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:40:49.719162 26847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:49.732654 26847 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:42661
I20260807 18:40:49.732780 27054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:42661 every 8 connection(s)
I20260807 18:40:49.733898 26847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:40:49.743907 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 26847
I20260807 18:40:49.744024 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:40:49.744343 6697 external_mini_cluster.cc:1428] Setting key b94890f633c69d13b528775755d0234b
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:49.751736 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:49.736047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:50145 (local address 127.6.138.126:46133)
0807 18:40:49.736207 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:49.736212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:49.737319 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 18:40:49.738581 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:49.738588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:49.738649 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:49.738757 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:49.740126 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.740970 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.742668 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.743051 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.743161 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:49.747185 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:49.747189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:49.747192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:49.747220 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:49.749156 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:49.749826 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:49.749830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:49.749832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:49.749927 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:49.750383 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:49.750387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:49.750389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:49.750626 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:49.750728 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:49.751451 (+ 723us) server_negotiation.cc:299] Negotiation successful
0807 18:40:49.751581 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:40:49.752878 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:49.736596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:46133 (local address 127.6.138.67:50145)
0807 18:40:49.737159 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:49.737217 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:49.738312 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:49.738964 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:49.738973 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:49.739308 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:49.739950 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:49.739967 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.741116 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.741120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:49.742498 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:49.742509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:49.743207 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:49.743211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:49.743398 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:49.745098 (+ 1700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:49.745125 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:49.746975 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:49.749305 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:49.749312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:49.749330 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:49.749699 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:49.750063 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:49.750066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:49.750068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:49.750266 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:49.750760 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:49.750767 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:49.751123 (+ 356us) client_negotiation.cc:769] Sending connection context
0807 18:40:49.751377 (+ 254us) client_negotiation.cc:240] Negotiation successful
0807 18:40:49.751733 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":148,"threads_started":1}
I20260807 18:40:49.754199 27056 heartbeater.cc:347] Connected to a master server at 127.6.138.126:46133
I20260807 18:40:49.754451 27056 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:49.755000 27056 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
I20260807 18:40:49.756186 26261 ts_manager.cc:194] Registered new tserver with Master: 2550682d0a8e47e8aea2d45f1ff74caa (127.6.138.67:42661)
I20260807 18:40:49.756938 26261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:50145
I20260807 18:40:49.760289 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:40:49.761133 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 26406
I20260807 18:40:49.784790 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 26633
I20260807 18:40:49.811399 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 26847
I20260807 18:40:49.822896 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 26194
I20260807 18:40:49.839565 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:46133
--webserver_interface=127.6.138.126
--webserver_port=40723
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:46133
--ranger_config_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:51107/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.138.84:51107/kms with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:40:49.956166 27060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:49.956465 27060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:49.956506 27060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:49.960670 27060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:40:49.960731 27060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:49.960752 27060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:49.960769 27060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:40:49.960784 27060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:40:49.966499 27060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.138.84:51107/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:46133
--ranger_config_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:46133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=40723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:49.967546 27060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:49.968622 27060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:49.977908 27068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:49.977944 27070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:49.977926 27066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:49.978433 27060 server_base.cc:1034] running on GCE node
I20260807 18:40:49.978929 27060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:49.979962 27060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:49.981156 27060 hybrid_clock.cc:648] HybridClock initialized: now 1786128049981116 us; error 51 us; skew 500 ppm
Aug 07 18:40:49 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:49.983986 27060 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:40:49.984943 27060 webserver.cc:460] Webserver started at http://127.6.138.126:40723/ using document root <none> and password file <none>
I20260807 18:40:49.985523 27060 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:49.985630 27060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:40:50 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128049, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:50.067903 27060 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:40:50.070458 27088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:50.071817 27060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:40:50.071977 27060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "2d2e8dfc8f0e481ebef10a01bea0f779"
format_stamp: "Formatted at 2026-08-07 18:40:47 on dist-test-slave-2d19"
server_key: "1bebc100e7be98bd66d34558b3d8ff7e"
server_key_iv: "d051f3bf0d7ca29fde8df30fcc06bb86"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:50.072610 27060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:50.086186 27060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:50.089334 27060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:50.089861 27060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:50.096989 27060 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:46133
I20260807 18:40:50.096993 27156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:46133 every 8 connection(s)
I20260807 18:40:50.098153 27060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:40:50.102494 27157 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:40:50.104200 27157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Bootstrap starting.
I20260807 18:40:50.108593 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 27060
I20260807 18:40:50.109160 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:51107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:39825
--local_ip_for_outbound_sockets=127.6.138.65
--tserver_master_addrs=127.6.138.126:46133
--webserver_port=44553
--webserver_interface=127.6.138.65
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.138.84:51107/kms with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:40:50.123297 27157 log.cc:824] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:50.132016 27157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:40:50.132553 27157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Bootstrap complete.
I20260807 18:40:50.138168 27157 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } }
I20260807 18:40:50.138695 27157 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2e8dfc8f0e481ebef10a01bea0f779, State: Initialized, Role: FOLLOWER
I20260807 18:40:50.139256 27157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } }
I20260807 18:40:50.139402 27157 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:40:50.139484 27157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:40:50.139582 27157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:40:50.142149 27157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } }
I20260807 18:40:50.142518 27157 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d2e8dfc8f0e481ebef10a01bea0f779; no voters:
I20260807 18:40:50.142875 27157 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:40:50.143023 27168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:40:50.143306 27168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [term 2 LEADER]: Becoming Leader. State: Replica: 2d2e8dfc8f0e481ebef10a01bea0f779, State: Running, Role: LEADER
I20260807 18:40:50.143657 27168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } }
I20260807 18:40:50.144145 27157 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:40:50.145143 27172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } } }
I20260807 18:40:50.145267 27172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: This master's current role is: LEADER
I20260807 18:40:50.146076 27186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:40:50.146075 27176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2e8dfc8f0e481ebef10a01bea0f779. Latest consensus state: current_term: 2 leader_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d2e8dfc8f0e481ebef10a01bea0f779" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 46133 } } }
I20260807 18:40:50.146217 27176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779 [sys.catalog]: This master's current role is: LEADER
I20260807 18:40:50.146551 27157 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:40:50.153620 27186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:40:50.154526 27186 catalog_manager.cc:1259] Loaded cluster ID: 83fd58670e63432c98fa1d44a5b979dd
I20260807 18:40:50.154642 27186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:40:50.157330 27186 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:40:50.158126 27186 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Loaded TSK: 0
I20260807 18:40:50.159443 27186 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:40:50.264691 27166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:50.265116 27166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:50.265264 27166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:50.269838 27166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:50.269924 27166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:50.270040 27166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:40:50.275779 27166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.138.84:51107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:39825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=44553
--tserver_master_addrs=127.6.138.126:46133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:50.277076 27166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:50.278421 27166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:50.289456 27230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:50.289466 27231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:50.290073 27233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:50.293176 27166 server_base.cc:1034] running on GCE node
I20260807 18:40:50.294216 27166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:50.294989 27166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:50.296193 27166 hybrid_clock.cc:648] HybridClock initialized: now 1786128050296158 us; error 49 us; skew 500 ppm
Aug 07 18:40:50 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128050, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:50.300026 27166 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:40:50.301157 27166 webserver.cc:460] Webserver started at http://127.6.138.65:44553/ using document root <none> and password file <none>
I20260807 18:40:50.301786 27166 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:50.301921 27166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:40:50 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128050, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:50.327312 27166 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 18:40:50.330442 27244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:50.331905 27166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:40:50.332055 27166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46"
format_stamp: "Formatted at 2026-08-07 18:40:48 on dist-test-slave-2d19"
server_key: "e9705ccfe5bf3f23ec550486db600c9d"
server_key_iv: "7232e07150b6a2748301a7990dd37073"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:50.332772 27166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:50.360991 27166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:50.365021 27166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:50.365252 27166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:50.366076 27166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:40:50.367265 27166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:40:50.367350 27166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:50.367436 27166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:40:50.367475 27166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:50.379946 27166 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:39825
I20260807 18:40:50.381305 27166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:40:50.386416 27433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:39825 every 8 connection(s)
I20260807 18:40:50.389714 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 27166
I20260807 18:40:50.390223 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:51107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:36721
--local_ip_for_outbound_sockets=127.6.138.66
--tserver_master_addrs=127.6.138.126:46133
--webserver_port=45333
--webserver_interface=127.6.138.66
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.138.84:51107/kms with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:40:50 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128050, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:50.409502 27438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:50.388994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:46133 (local address 127.6.138.65:48229)
0807 18:40:50.391748 (+ 2754us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:50.391814 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:50.393018 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:50.395047 (+ 2029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:50.395057 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:50.395439 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:50.396090 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:50.396109 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:50.398058 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:50.398063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:50.399476 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:50.399489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:50.400860 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:50.400864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:50.401064 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:50.401809 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:50.401829 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:50.403332 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:50.405677 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:50.405684 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:50.405699 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:50.406083 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:50.406514 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:50.406518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:50.406520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:50.406694 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:50.407518 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:50.407524 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:50.407825 (+ 301us) client_negotiation.cc:769] Sending connection context
0807 18:40:50.408039 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 18:40:50.408422 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2514,"thread_start_us":154,"threads_started":1}
I20260807 18:40:50.411162 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:50.388957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:48229 (local address 127.6.138.126:46133)
0807 18:40:50.391389 (+ 2432us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:50.391417 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:50.392007 (+ 590us) server_negotiation.cc:407] Connection header received
0807 18:40:50.393608 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:50.393626 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:50.394108 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:50.394500 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:50.396336 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:50.397891 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:50.399647 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:50.400452 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:50.400622 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:50.403546 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:50.403551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:50.403566 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:50.403599 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:50.405544 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:50.406233 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:50.406240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:50.406246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:50.406396 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:50.406811 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:50.406814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:50.406816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:50.407319 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:50.407446 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:50.410058 (+ 2612us) server_negotiation.cc:299] Negotiation successful
0807 18:40:50.410372 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2228,"thread_start_us":130,"threads_started":1}
I20260807 18:40:50.414124 27434 heartbeater.cc:347] Connected to a master server at 127.6.138.126:46133
I20260807 18:40:50.414410 27434 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:50.414940 27434 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
W20260807 18:40:50.417313 27434 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:46133 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:40:50.417534 27434 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:50.417789 27434 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
I20260807 18:40:50.418728 27434 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:50.418951 27434 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
W20260807 18:40:50.419757 27434 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:40:50.565537 27439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:50.565958 27439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:50.566031 27439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:50.572415 27439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:50.572547 27439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:50.572695 27439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:40:50.581259 27439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.138.84:51107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:36721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=45333
--tserver_master_addrs=127.6.138.126:46133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:50.582796 27439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:50.584276 27439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:50.596879 27454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:50.597009 27448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:50.596880 27447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:50.598471 27439 server_base.cc:1034] running on GCE node
I20260807 18:40:50.599238 27439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:50.600508 27439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:50.604355 27439 hybrid_clock.cc:648] HybridClock initialized: now 1786128050604008 us; error 369 us; skew 500 ppm
Aug 07 18:40:50 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128050, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:50.608315 27439 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:40:50.609647 27439 webserver.cc:460] Webserver started at http://127.6.138.66:45333/ using document root <none> and password file <none>
I20260807 18:40:50.610507 27439 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:50.610699 27439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:40:50 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128050, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:50.655121 27439 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:40:50.658509 27463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:50.660327 27439 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:40:50.660472 27439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "af8394accc02427fb11162a43b3401da"
format_stamp: "Formatted at 2026-08-07 18:40:49 on dist-test-slave-2d19"
server_key: "6e21353ffd63c3b0b29824d6e468f904"
server_key_iv: "83dac9837c9bfe76c547759104ab06bd"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:50.661276 27439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:50.687423 27439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:50.691313 27439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:50.691545 27439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:50.692378 27439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:40:50.693522 27439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:40:50.693602 27439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:50.693673 27439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:40:50.693706 27439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:50.707413 27439 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:36721
I20260807 18:40:50.707490 27657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:36721 every 8 connection(s)
I20260807 18:40:50.708922 27439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:40:50.717021 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 27439
I20260807 18:40:50.717533 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.138.84:51107/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:42661
--local_ip_for_outbound_sockets=127.6.138.67
--tserver_master_addrs=127.6.138.126:46133
--webserver_port=39383
--webserver_interface=127.6.138.67
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.138.84:51107/kms with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:40:50 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128050, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:50.726161 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:50.711127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:37835 (local address 127.6.138.126:46133)
0807 18:40:50.711300 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:50.711305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:50.712484 (+ 1179us) server_negotiation.cc:407] Connection header received
0807 18:40:50.713915 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:50.713919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:50.713986 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:50.714097 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:50.715584 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:50.716125 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:50.717959 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:50.718370 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:50.718522 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:50.722052 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:50.722057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:50.722060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:50.722101 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:50.723946 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:50.724548 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:50.724551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:50.724553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:50.724642 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:50.725003 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:50.725006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:50.725008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:50.725210 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:50.725321 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:50.725841 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 18:40:50.725974 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:40:50.727057 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:50.711647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:46133 (local address 127.6.138.66:37835)
0807 18:40:50.712307 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:50.712366 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:50.713600 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:50.714322 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:50.714332 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:50.714694 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:50.715398 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:50.715423 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:50.716312 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:50.716317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:50.717762 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:50.717775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:50.718751 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:50.718758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:50.718985 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:50.720021 (+ 1036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:50.720043 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:50.721838 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:50.724060 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:50.724066 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:50.724083 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:50.724437 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:50.724738 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:50.724742 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:50.724744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:50.724896 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:50.725321 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:50.725329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:50.725591 (+ 262us) client_negotiation.cc:769] Sending connection context
0807 18:40:50.725783 (+ 192us) client_negotiation.cc:240] Negotiation successful
0807 18:40:50.726111 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":186,"threads_started":1}
I20260807 18:40:50.728480 27659 heartbeater.cc:347] Connected to a master server at 127.6.138.126:46133
I20260807 18:40:50.728736 27659 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:50.729269 27659 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
W20260807 18:40:50.731019 27659 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:46133 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:40:50.731142 27659 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:50.731309 27659 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
I20260807 18:40:50.732236 27659 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:50.732427 27659 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
W20260807 18:40:50.733202 27659 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:40:50.880811 27669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:50.881330 27669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:50.881476 27669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:50.885905 27669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:50.886051 27669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:50.886147 27669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:40:50.892434 27669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.138.84:51107/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:42661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=39383
--tserver_master_addrs=127.6.138.126:46133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:50.893599 27669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:50.894660 27669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:50.905930 27680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:50.906471 27675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:50.906595 27676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:50.906744 27669 server_base.cc:1034] running on GCE node
I20260807 18:40:50.907290 27669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:50.908133 27669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:50.909374 27669 hybrid_clock.cc:648] HybridClock initialized: now 1786128050909340 us; error 47 us; skew 500 ppm
Aug 07 18:40:50 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128050, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:50.912981 27669 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:40:50.913865 27669 webserver.cc:460] Webserver started at http://127.6.138.67:39383/ using document root <none> and password file <none>
I20260807 18:40:50.914502 27669 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:50.914628 27669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:40:50 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128050, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:50.957378 27669 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.003s sys 0.000s
I20260807 18:40:50.962035 27693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:50.963811 27669 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260807 18:40:50.963948 27669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "2550682d0a8e47e8aea2d45f1ff74caa"
format_stamp: "Formatted at 2026-08-07 18:40:49 on dist-test-slave-2d19"
server_key: "9362badc19ecb7399f025d7d7ffa0961"
server_key_iv: "2ee1a0ec3fe1a340d5ea740c0cdc9621"
server_key_version: "kuduclusterkey@0"
I20260807 18:40:50.964749 27669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:50.985675 27669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:50.989459 27669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:50.989782 27669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:50.990612 27669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:40:50.991662 27669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:40:50.991772 27669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:50.991856 27669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:40:50.991905 27669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:51.004237 27669 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:42661
I20260807 18:40:51.004333 27888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:42661 every 8 connection(s)
I20260807 18:40:51.005893 27669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:40:51.008848 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 27669
I20260807 18:40:51.021415 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.011508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.6.138.126:46133)
0807 18:40:51.012547 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:51.012556 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:51.012571 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:40:51.012621 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:51.012624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:51.012679 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:51.012778 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:51.013622 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.014422 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.015357 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.015686 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.015803 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.017550 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:51.017555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:51.017558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:51.017588 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:51.019519 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:51.020034 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:51.020038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:51.020039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:51.020132 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:51.020491 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:51.020495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:51.020497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:51.020715 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:51.020914 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:51.021092 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 18:40:51.021219 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":98,"threads_started":1}
Aug 07 18:40:51 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128050, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:51.030570 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.011403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:57713 (local address 127.6.138.126:46133)
0807 18:40:51.011739 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:51.011744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:51.011760 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:40:51.014373 (+ 2613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:51.014376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:51.014422 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:51.014474 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:51.017771 (+ 3297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.019525 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.021429 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.021770 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.021831 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.026452 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:51.026455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:51.026460 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:51.026487 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:51.028220 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:51.028836 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:51.028841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:51.028845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:51.028921 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:51.029288 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:51.029293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:51.029297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:51.029509 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:51.029580 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:51.030223 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 18:40:51.030350 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268}
I20260807 18:40:51.032290 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.010195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:46133 (local address 127.6.138.67:57713)
0807 18:40:51.011318 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:51.011417 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:51.014135 (+ 2718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:51.016523 (+ 2388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:51.016534 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:51.016912 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:51.017625 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:51.017642 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.019939 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.019945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:51.021289 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:51.021301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.022128 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.022132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:51.023150 (+ 1018us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.023821 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:51.023850 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:51.026250 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:51.028363 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:51.028371 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:51.028390 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:51.028727 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:51.029013 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:51.029018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:51.029022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:51.029196 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:51.029641 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:51.029650 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:51.029951 (+ 301us) client_negotiation.cc:769] Sending connection context
0807 18:40:51.030878 (+ 927us) client_negotiation.cc:240] Negotiation successful
0807 18:40:51.031226 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":157,"threads_started":1}
I20260807 18:40:51.033540 27889 heartbeater.cc:347] Connected to a master server at 127.6.138.126:46133
I20260807 18:40:51.033797 27889 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:51.034409 27889 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
I20260807 18:40:51.035929 27107 ts_manager.cc:194] Registered new tserver with Master: 2550682d0a8e47e8aea2d45f1ff74caa (127.6.138.67:42661)
I20260807 18:40:51.037578 27107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:57713
Aug 07 18:40:51 dist-test-slave-2d19 krb5kdc[25456](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786128051, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:51.420025 27434 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:51.420362 27434 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
I20260807 18:40:51.421066 27107 ts_manager.cc:194] Registered new tserver with Master: 0f248c4d9eaf43e9b0cbdabaedf1fa46 (127.6.138.65:39825)
I20260807 18:40:51.421833 27107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:48229
I20260807 18:40:51.733426 27659 heartbeater.cc:464] Registering TS with master...
I20260807 18:40:51.733736 27659 heartbeater.cc:511] Master 127.6.138.126:46133 requested a full tablet report, sending...
I20260807 18:40:51.734462 27107 ts_manager.cc:194] Registered new tserver with Master: af8394accc02427fb11162a43b3401da (127.6.138.66:36721)
I20260807 18:40:51.735069 27107 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:37835
I20260807 18:40:51.743134 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:40:51.755577 27916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.744788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.6.138.126:46133)
0807 18:40:51.745558 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:51.745563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:51.745576 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:40:51.745613 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:51.745616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:51.745663 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:51.745763 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:51.746807 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.747627 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.748927 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.749279 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.749373 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.751224 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:51.751228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:51.751231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:51.751258 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:51.753202 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:51.753703 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:51.753707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:51.753709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:51.753802 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:51.754139 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:51.754143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:51.754145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:51.754372 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:51.754504 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:51.755266 (+ 762us) server_negotiation.cc:299] Negotiation successful
0807 18:40:51.755401 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":106,"threads_started":1}
I20260807 18:40:51.759867 27107 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:40:51.764585 27107 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 07 18:40:51 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128051, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:51.791301 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.779698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:36721 (local address 127.0.0.1:51360)
0807 18:40:51.783347 (+ 3649us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:51.783412 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:51.783554 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:51.785600 (+ 2046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:51.785606 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:51.785635 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:51.785803 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:51.785811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.787383 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.787387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:51.789014 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:51.789027 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.790529 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.790534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:51.790707 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.790761 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:40:51.790926 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 18:40:51.791081 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3352,"thread_start_us":119,"threads_started":1}
I20260807 18:40:51.791998 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.780553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.6.138.66:36721)
0807 18:40:51.780885 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:51.780893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:51.783525 (+ 2632us) server_negotiation.cc:407] Connection header received
0807 18:40:51.783717 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:51.783723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:51.785406 (+ 1683us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:51.785582 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:51.785974 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.787225 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.789171 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.790100 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.790232 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.790415 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:51.791679 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0807 18:40:51.791818 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":122,"threads_started":1}
I20260807 18:40:51.796001 27930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.781655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:39825 (local address 127.0.0.1:48304)
0807 18:40:51.788502 (+ 6847us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:51.788518 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:51.788616 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:51.789627 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:51.789630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:51.789649 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:51.790570 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:51.790578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.792127 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.792130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:51.793403 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:51.793415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.795603 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.795607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:51.795705 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.795729 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:40:51.795798 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 18:40:51.795866 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6767,"thread_start_us":97,"threads_started":1}
I20260807 18:40:51.796450 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.782089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.6.138.65:39825)
0807 18:40:51.782382 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:51.782389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:51.788969 (+ 6580us) server_negotiation.cc:407] Connection header received
0807 18:40:51.789067 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:51.789073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:51.789323 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:51.789475 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:51.790719 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.791977 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.793552 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.795288 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.795390 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.795539 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:51.796194 (+ 655us) server_negotiation.cc:299] Negotiation successful
0807 18:40:51.796318 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":78,"threads_started":1}
I20260807 18:40:51.799732 27320 tablet_service.cc:1467] Processing CreateTablet for tablet 66de54a4fc98469da29b4d3d106d073d (DEFAULT_TABLE table=test-table [id=c876c5e1c51a48e09c191439324a0e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:40:51.800848 27320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66de54a4fc98469da29b4d3d106d073d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:51.796175 27535 tablet_service.cc:1467] Processing CreateTablet for tablet 66de54a4fc98469da29b4d3d106d073d (DEFAULT_TABLE table=test-table [id=c876c5e1c51a48e09c191439324a0e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:40:51.804348 27535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66de54a4fc98469da29b4d3d106d073d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:51.804411 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.782510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:42661 (local address 127.0.0.1:52980)
0807 18:40:51.788783 (+ 6273us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:51.788795 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:51.788858 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:51.797369 (+ 8511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:51.797372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:51.797391 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:51.797483 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:51.797489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.799028 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.799031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:51.801395 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:51.801407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.802480 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.802483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:51.802584 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.802610 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:40:51.802688 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 18:40:51.802760 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6223,"thread_start_us":87,"threads_started":1}
I20260807 18:40:51.805028 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:51.782657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.6.138.67:42661)
0807 18:40:51.796791 (+ 14134us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:51.796798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:51.796824 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:40:51.796905 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:51.796910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:51.797134 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:51.797636 (+ 502us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:51.797772 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.798892 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.801541 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:51.802336 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:51.802454 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:51.802596 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:51.804780 (+ 2184us) server_negotiation.cc:299] Negotiation successful
0807 18:40:51.804892 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14012,"thread_start_us":125,"threads_started":1}
I20260807 18:40:51.808034 27769 tablet_service.cc:1467] Processing CreateTablet for tablet 66de54a4fc98469da29b4d3d106d073d (DEFAULT_TABLE table=test-table [id=c876c5e1c51a48e09c191439324a0e78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:40:51.809157 27769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66de54a4fc98469da29b4d3d106d073d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:51.820168 27935 tablet_bootstrap.cc:492] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46: Bootstrap starting.
I20260807 18:40:51.820840 27932 tablet_bootstrap.cc:492] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da: Bootstrap starting.
I20260807 18:40:51.823360 27932 tablet_bootstrap.cc:654] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:51.823478 27935 tablet_bootstrap.cc:654] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:51.824445 27935 log.cc:824] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:51.824442 27932 log.cc:824] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:51.826609 27935 tablet_bootstrap.cc:492] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46: No bootstrap required, opened a new log
I20260807 18:40:51.826813 27935 ts_tablet_manager.cc:1397] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260807 18:40:51.828625 27934 tablet_bootstrap.cc:492] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: Bootstrap starting.
I20260807 18:40:51.831228 27934 tablet_bootstrap.cc:654] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:51.832202 27934 log.cc:824] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:51.831650 27935 raft_consensus.cc:348] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } }
I20260807 18:40:51.832593 27935 raft_consensus.cc:374] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:51.832691 27935 raft_consensus.cc:729] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f248c4d9eaf43e9b0cbdabaedf1fa46, State: Initialized, Role: FOLLOWER
I20260807 18:40:51.833199 27935 consensus_queue.cc:260] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } }
I20260807 18:40:51.834270 27935 ts_tablet_manager.cc:1428] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260807 18:40:51.834649 27434 heartbeater.cc:503] Master 127.6.138.126:46133 was elected leader, sending a full tablet report...
I20260807 18:40:51.838929 27932 tablet_bootstrap.cc:492] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da: No bootstrap required, opened a new log
I20260807 18:40:51.839188 27932 ts_tablet_manager.cc:1397] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260807 18:40:51.844007 27932 raft_consensus.cc:348] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } }
I20260807 18:40:51.844694 27932 raft_consensus.cc:374] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:51.844805 27932 raft_consensus.cc:729] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af8394accc02427fb11162a43b3401da, State: Initialized, Role: FOLLOWER
I20260807 18:40:51.845369 27932 consensus_queue.cc:260] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } }
I20260807 18:40:51.846654 27932 ts_tablet_manager.cc:1428] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260807 18:40:51.847009 27659 heartbeater.cc:503] Master 127.6.138.126:46133 was elected leader, sending a full tablet report...
I20260807 18:40:51.847074 27934 tablet_bootstrap.cc:492] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: No bootstrap required, opened a new log
I20260807 18:40:51.847286 27934 ts_tablet_manager.cc:1397] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260807 18:40:51.852865 27934 raft_consensus.cc:348] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } }
I20260807 18:40:51.853181 27934 raft_consensus.cc:374] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:51.853260 27934 raft_consensus.cc:729] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2550682d0a8e47e8aea2d45f1ff74caa, State: Initialized, Role: FOLLOWER
I20260807 18:40:51.853967 27934 consensus_queue.cc:260] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } }
I20260807 18:40:51.854719 27889 heartbeater.cc:503] Master 127.6.138.126:46133 was elected leader, sending a full tablet report...
I20260807 18:40:51.857336 27934 ts_tablet_manager.cc:1428] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
W20260807 18:40:51.888051 27436 tablet.cc:2369] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:40:51.961318 27661 tablet.cc:2369] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:40:52.009008 27892 tablet.cc:2369] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 18:40:52 dist-test-slave-2d19 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128051, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for HTTP/127.6.138.84@KRBTEST.COM
I20260807 18:40:52.142095 27944 raft_consensus.cc:482] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:40:52.142359 27944 raft_consensus.cc:504] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } }
I20260807 18:40:52.143903 27944 leader_election.cc:288] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af8394accc02427fb11162a43b3401da (127.6.138.66:36721), 0f248c4d9eaf43e9b0cbdabaedf1fa46 (127.6.138.65:39825)
I20260807 18:40:52.151602 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:52.145638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:56279 (local address 127.6.138.65:39825)
0807 18:40:52.145814 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:52.145818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:52.145830 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:40:52.145878 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:52.145881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:52.145925 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:52.145989 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:52.147765 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.148496 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.150361 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.150879 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.150977 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:52.151024 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:52.151384 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 18:40:52.151466 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:40:52.152334 27948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:52.144899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:39825 (local address 127.6.138.67:56279)
0807 18:40:52.145191 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:52.145203 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:52.145295 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:52.147498 (+ 2203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:52.147501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:52.147524 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:52.147616 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:52.147622 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.148776 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.148779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:52.150220 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:52.150230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.151136 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.151140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:52.151239 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:52.151260 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:40:52.152060 (+ 800us) client_negotiation.cc:240] Negotiation successful
0807 18:40:52.152152 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":103,"threads_started":1}
I20260807 18:40:52.153250 27354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66de54a4fc98469da29b4d3d106d073d" candidate_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" is_pre_election: true
I20260807 18:40:52.153638 27354 raft_consensus.cc:2454] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2550682d0a8e47e8aea2d45f1ff74caa in term 0.
I20260807 18:40:52.154389 27694 leader_election.cc:302] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f248c4d9eaf43e9b0cbdabaedf1fa46, 2550682d0a8e47e8aea2d45f1ff74caa; no voters:
I20260807 18:40:52.154732 27944 raft_consensus.cc:2781] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:40:52.154814 27944 raft_consensus.cc:482] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:40:52.154855 27944 raft_consensus.cc:3037] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:52.156412 27944 raft_consensus.cc:504] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } }
I20260807 18:40:52.156780 27944 leader_election.cc:288] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [CANDIDATE]: Term 1 election: Requested vote from peers af8394accc02427fb11162a43b3401da (127.6.138.66:36721), 0f248c4d9eaf43e9b0cbdabaedf1fa46 (127.6.138.65:39825)
I20260807 18:40:52.157544 27354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66de54a4fc98469da29b4d3d106d073d" candidate_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46"
I20260807 18:40:52.157680 27354 raft_consensus.cc:3037] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:52.159082 27354 raft_consensus.cc:2454] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2550682d0a8e47e8aea2d45f1ff74caa in term 1.
I20260807 18:40:52.159543 27694 leader_election.cc:302] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f248c4d9eaf43e9b0cbdabaedf1fa46, 2550682d0a8e47e8aea2d45f1ff74caa; no voters:
I20260807 18:40:52.159802 27944 raft_consensus.cc:2781] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:40:52.160050 27944 raft_consensus.cc:686] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 1 LEADER]: Becoming Leader. State: Replica: 2550682d0a8e47e8aea2d45f1ff74caa, State: Running, Role: LEADER
I20260807 18:40:52.160562 27944 consensus_queue.cc:237] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } }
I20260807 18:40:52.164235 27107 catalog_manager.cc:5495] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa reported cstate change: term changed from 0 to 1, leader changed from <none> to 2550682d0a8e47e8aea2d45f1ff74caa (127.6.138.67). New cstate: current_term: 1 leader_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 42661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 } health_report { overall_health: UNKNOWN } } }
I20260807 18:40:52.168845 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:52.145461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:56145 (local address 127.6.138.66:36721)
0807 18:40:52.145609 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:52.145614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:52.145628 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:40:52.145686 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:52.145692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:52.145744 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:52.145818 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:40:52.146218 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.146908 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.149299 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.149885 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.149978 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:52.150031 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:52.168585 (+ 18554us) server_negotiation.cc:299] Negotiation successful
0807 18:40:52.168707 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:40:52.169277 27947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:52.144464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:36721 (local address 127.6.138.67:56145)
0807 18:40:52.144795 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:52.144812 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:52.144917 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:52.145944 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:52.145949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:52.145975 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:40:52.146085 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:52.146092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.147053 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.147057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:52.149152 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:52.149162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.164165 (+ 15003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.164169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:52.168356 (+ 4187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:52.168392 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:40:52.169002 (+ 610us) client_negotiation.cc:240] Negotiation successful
0807 18:40:52.169119 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":136,"threads_started":1}
I20260807 18:40:52.170953 27579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66de54a4fc98469da29b4d3d106d073d" candidate_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af8394accc02427fb11162a43b3401da" is_pre_election: true
I20260807 18:40:52.171331 27579 raft_consensus.cc:2454] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2550682d0a8e47e8aea2d45f1ff74caa in term 0.
I20260807 18:40:52.171712 27582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66de54a4fc98469da29b4d3d106d073d" candidate_uuid: "2550682d0a8e47e8aea2d45f1ff74caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af8394accc02427fb11162a43b3401da"
I20260807 18:40:52.171826 27582 raft_consensus.cc:3037] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:52.173424 27582 raft_consensus.cc:2454] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2550682d0a8e47e8aea2d45f1ff74caa in term 1.
I20260807 18:40:52.246546 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:52.240859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52988 (local address 127.6.138.67:42661)
0807 18:40:52.240992 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:52.240997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:52.241043 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:40:52.241258 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:52.241261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:52.241335 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:52.241478 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:40:52.241818 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.242626 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.244003 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:52.244400 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:52.244510 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:52.245062 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:40:52.245648 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:40:52.245778 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:52.246302 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 18:40:52.246367 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:40:52.254383 27354 raft_consensus.cc:1261] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [term 1 FOLLOWER]: Refusing update from remote peer 2550682d0a8e47e8aea2d45f1ff74caa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:40:52.255211 27952 consensus_queue.cc:1035] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f248c4d9eaf43e9b0cbdabaedf1fa46" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:40:52.261258 27582 raft_consensus.cc:1261] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [term 1 FOLLOWER]: Refusing update from remote peer 2550682d0a8e47e8aea2d45f1ff74caa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:40:52.261925 27952 consensus_queue.cc:1035] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [LEADER]: Connected to new peer: Peer: permanent_uuid: "af8394accc02427fb11162a43b3401da" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:40:52.272167 27961 mvcc.cc:205] Tried to move back new op lower bound from 7315980502025904128 to 7315980501653487616. Current Snapshot: MvccSnapshot[applied={T|T < 7315980502025904128}]
I20260807 18:40:52.276453 27962 mvcc.cc:205] Tried to move back new op lower bound from 7315980502025904128 to 7315980501653487616. Current Snapshot: MvccSnapshot[applied={T|T < 7315980502025904128}]
I20260807 18:40:52.280915 27964 mvcc.cc:205] Tried to move back new op lower bound from 7315980502025904128 to 7315980501653487616. Current Snapshot: MvccSnapshot[applied={T|T < 7315980502025904128}]
I20260807 18:40:52.286610 27107 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:40:52.286800 27107 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:40:52.289708 27107 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d2e8dfc8f0e481ebef10a01bea0f779: Sending DeleteTablet for 3 replicas of tablet 66de54a4fc98469da29b4d3d106d073d
I20260807 18:40:52.290581 27320 tablet_service.cc:1514] Processing DeleteTablet for tablet 66de54a4fc98469da29b4d3d106d073d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:40:52 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:48304
I20260807 18:40:52.290653 27769 tablet_service.cc:1514] Processing DeleteTablet for tablet 66de54a4fc98469da29b4d3d106d073d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:40:52 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:52980
I20260807 18:40:52.291123 27974 tablet_replica.cc:330] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46: stopping tablet replica
I20260807 18:40:52.291149 27535 tablet_service.cc:1514] Processing DeleteTablet for tablet 66de54a4fc98469da29b4d3d106d073d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:40:52 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:51360
I20260807 18:40:52.291601 27974 raft_consensus.cc:2229] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:40:52.291957 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 27166
I20260807 18:40:52.292083 27974 raft_consensus.cc:2258] T 66de54a4fc98469da29b4d3d106d073d P 0f248c4d9eaf43e9b0cbdabaedf1fa46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:40:52.308538 27978 tablet_replica.cc:330] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: stopping tablet replica
I20260807 18:40:52.308946 27978 raft_consensus.cc:2229] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:40:52.309398 27978 raft_consensus.cc:2258] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 18:40:52.309492 27089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:39825 (error 108)
I20260807 18:40:52.309798 27979 tablet_replica.cc:330] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da: stopping tablet replica
I20260807 18:40:52.310105 27979 raft_consensus.cc:2229] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:40:52.310438 27979 raft_consensus.cc:2258] T 66de54a4fc98469da29b4d3d106d073d P af8394accc02427fb11162a43b3401da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:40:52.310896 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 27439
I20260807 18:40:52.312386 27978 ts_tablet_manager.cc:1905] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:40:52.315972 27978 ts_tablet_manager.cc:1918] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:40:52.316102 27978 log.cc:1196] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/66de54a4fc98469da29b4d3d106d073d
I20260807 18:40:52.316470 27978 ts_tablet_manager.cc:1939] T 66de54a4fc98469da29b4d3d106d073d P 2550682d0a8e47e8aea2d45f1ff74caa: Deleting consensus metadata
I20260807 18:40:52.328727 27984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:52.318743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:39825 (local address 127.0.0.1:48308)
0807 18:40:52.328520 (+ 9777us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:52.328613 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:39825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9690,"thread_start_us":69,"threads_started":1}
W20260807 18:40:52.328919 27089 catalog_manager.cc:4584] TS 0f248c4d9eaf43e9b0cbdabaedf1fa46 (127.6.138.65:39825): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 66de54a4fc98469da29b4d3d106d073d: Network error: Client connection negotiation failed: client connection to 127.6.138.65:39825: connect: Connection refused (error 111)
I20260807 18:40:52.329103 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 27669
W20260807 18:40:52.329449 27984 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:40:52.327306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:36721 (local address 127.0.0.1:51366)
0807 18:40:52.329259 (+ 1953us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:52.329321 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.66:36721: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":1874,"thread_start_us":68,"threads_started":1}
I20260807 18:40:52.346264 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 27060
I20260807 18:40:52.346328 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:52.345978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:42661 (local address 127.0.0.1:52994)
0807 18:40:52.346140 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:52.346214 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.67:42661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 18:40:52.360771 6697 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:40:52.708233 6697 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:40:52.708390 6697 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:40:53.064074 6697 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:40:53.064 UTC [25518] LOG: received smart shutdown request
2026-08-07 18:40:53.070 UTC [25518] LOG: background worker "logical replication launcher" (PID 25534) exited with exit code 1
2026-08-07 18:40:53.071 UTC [25524] LOG: shutting down
2026-08-07 18:40:53.261 UTC [25518] LOG: database system is shut down
2026-08-07T18:40:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51594 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:40:53 dist-test-slave-2d19 krb5kdc[28001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:40:53 dist-test-slave-2d19 krb5kdc[28001](info): set up 2 sockets
Aug 07 18:40:53 dist-test-slave-2d19 krb5kdc[28001](info): commencing operation
krb5kdc: starting...
W20260807 18:40:54.677820 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:40:54 dist-test-slave-2d19 krb5kdc[28001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:40:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:40:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:40:54.813357 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42433
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42433,127.6.138.125:46065,127.6.138.124:38933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:40:54.927819 28020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:54.928099 28020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:54.928140 28020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:54.932176 28020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:40:54.932243 28020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:54.932279 28020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:54.932296 28020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:40:54.932312 28020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:40:54.937631 28020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42433,127.6.138.125:46065,127.6.138.124:38933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:54.938683 28020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:54.939668 28020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:54.948336 28026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:54.948532 28020 server_base.cc:1034] running on GCE node
W20260807 18:40:54.948331 28025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:54.948340 28029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:54.949306 28020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:54.950482 28020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:54.951669 28020 hybrid_clock.cc:648] HybridClock initialized: now 1786128054951634 us; error 43 us; skew 500 ppm
Aug 07 18:40:54 dist-test-slave-2d19 krb5kdc[28001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128054, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:54.954751 28020 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:40:54.955694 28020 webserver.cc:460] Webserver started at http://127.6.138.126:36771/ using document root <none> and password file <none>
I20260807 18:40:54.956332 28020 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:54.956399 28020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:40:54.956637 28020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:40:54.958698 28020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "2b4b1285573545c8873e229acac66e23"
format_stamp: "Formatted at 2026-08-07 18:40:54 on dist-test-slave-2d19"
server_key: "feecbf15ccfa48d3ca90d2da6fd8b652"
server_key_iv: "d0ee7bce6642bcb2f8e4086c1d20f4fb"
server_key_version: "encryptionkey@0"
I20260807 18:40:54.959275 28020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "2b4b1285573545c8873e229acac66e23"
format_stamp: "Formatted at 2026-08-07 18:40:54 on dist-test-slave-2d19"
server_key: "feecbf15ccfa48d3ca90d2da6fd8b652"
server_key_iv: "d0ee7bce6642bcb2f8e4086c1d20f4fb"
server_key_version: "encryptionkey@0"
I20260807 18:40:54.963129 28020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260807 18:40:54.965648 28036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:54.966989 28020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:40:54.967160 28020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "2b4b1285573545c8873e229acac66e23"
format_stamp: "Formatted at 2026-08-07 18:40:54 on dist-test-slave-2d19"
server_key: "feecbf15ccfa48d3ca90d2da6fd8b652"
server_key_iv: "d0ee7bce6642bcb2f8e4086c1d20f4fb"
server_key_version: "encryptionkey@0"
I20260807 18:40:54.967285 28020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:54.985641 28020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:54.989111 28020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:54.989336 28020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:54.996551 28020 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:42433
I20260807 18:40:54.996587 28124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:42433 every 8 connection(s)
I20260807 18:40:54.997524 28020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:40:55.000464 28126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:55.000980 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 28020
I20260807 18:40:55.001130 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:40:55.001446 6697 external_mini_cluster.cc:1428] Setting key d4c6953fe6d062f9e0baf8f045f29c78
I20260807 18:40:55.005112 28126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.138.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.125@KRBTEST.COM" created.
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128054, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:55.021939 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.007643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42433 (local address 127.0.0.1:36670)
0807 18:40:55.008159 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.008213 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:55.009092 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:55.009948 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:55.009953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:55.010284 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:55.010548 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.010561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.012250 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.012313 (+ 63us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.013232 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.013243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.013910 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.013914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.014046 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.014751 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:55.014769 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:55.016391 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:55.018784 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.018794 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.018806 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.019038 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.019387 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.019390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.019391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.019507 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.020119 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:55.020125 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:55.020324 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 18:40:55.020509 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 18:40:55.020916 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":108,"thread_start_us":155,"threads_started":1}
I20260807 18:40:55.021939 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.007643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.6.138.126:42433)
0807 18:40:55.008231 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.008246 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.008328 (+ 82us) server_negotiation.cc:407] Connection header received
0807 18:40:55.009438 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.009448 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.009723 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.009950 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.010728 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.012101 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.013373 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.013748 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.013870 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.016548 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.016554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.016567 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.016594 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.018667 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.019176 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.019181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.019182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.019285 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.019612 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.019615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.019617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.019996 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.020145 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.020600 (+ 455us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.020860 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":451,"spinlock_wait_cycles":17280,"thread_start_us":158,"threads_started":1}
I20260807 18:40:55.024884 28126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:40:55.025655 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.025360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:46065 (local address 127.0.0.1:33034)
0807 18:40:55.025511 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.025586 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:40:55.025925 28042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
I20260807 18:40:55.026906 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.026706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:46065 (local address 127.0.0.1:33040)
0807 18:40:55.026809 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.026853 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 18:40:55.027100 28126 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.125:46065: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab.
Entry for principal kudu/127.6.138.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab.
I20260807 18:40:55.033218 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.125:46065
--webserver_interface=127.6.138.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab
--principal=kudu/127.6.138.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42433,127.6.138.125:46065,127.6.138.124:38933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:40:55.076342 28126 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } attempt: 1
I20260807 18:40:55.077332 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.077018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:46065 (local address 127.0.0.1:33052)
0807 18:40:55.077184 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.077250 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:40:55.078279 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.078015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:46065 (local address 127.0.0.1:33068)
0807 18:40:55.078141 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.078199 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:40:55.078433 28126 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.125:46065: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
I20260807 18:40:55.146617 28126 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } attempt: 2
I20260807 18:40:55.147455 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.147164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:46065 (local address 127.0.0.1:33070)
0807 18:40:55.147310 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.147379 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 18:40:55.148308 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.148018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:46065 (local address 127.0.0.1:33074)
0807 18:40:55.148157 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.148208 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 18:40:55.148504 28126 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.125:46065: Network error: Client connection negotiation failed: client connection to 127.6.138.125:46065: connect: Connection refused (error 111)
W20260807 18:40:55.149719 28134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:55.149991 28134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:55.150102 28134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:55.154229 28134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:40:55.154302 28134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:55.154323 28134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:55.154340 28134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:40:55.154387 28134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:40:55.160118 28134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42433,127.6.138.125:46065,127.6.138.124:38933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab
--principal=kudu/127.6.138.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.125:46065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.138.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:55.161404 28134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:55.162544 28134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:55.172315 28147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:55.172318 28145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:55.172304 28143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:55.172616 28134 server_base.cc:1034] running on GCE node
I20260807 18:40:55.173323 28134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:55.174461 28134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:55.175647 28134 hybrid_clock.cc:648] HybridClock initialized: now 1786128055175630 us; error 28 us; skew 500 ppm
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:55.178762 28134 init.cc:377] Logged in from keytab as kudu/127.6.138.125@KRBTEST.COM (short username kudu)
I20260807 18:40:55.179675 28134 webserver.cc:460] Webserver started at http://127.6.138.125:34069/ using document root <none> and password file <none>
I20260807 18:40:55.180336 28134 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:55.180403 28134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:40:55.180583 28134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:40:55.182638 28134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/data/instance:
uuid: "e9e1c8c891be41f0bec1441ad22eb257"
format_stamp: "Formatted at 2026-08-07 18:40:55 on dist-test-slave-2d19"
server_key: "c6ad38426f864d131fc73d22e370ed18"
server_key_iv: "4b731ae9e717eb1006e3fc75680cd691"
server_key_version: "encryptionkey@0"
I20260807 18:40:55.183250 28134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/wal/instance:
uuid: "e9e1c8c891be41f0bec1441ad22eb257"
format_stamp: "Formatted at 2026-08-07 18:40:55 on dist-test-slave-2d19"
server_key: "c6ad38426f864d131fc73d22e370ed18"
server_key_iv: "4b731ae9e717eb1006e3fc75680cd691"
server_key_version: "encryptionkey@0"
I20260807 18:40:55.187000 28134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:40:55.189642 28157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:55.190981 28134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:40:55.191131 28134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/wal
uuid: "e9e1c8c891be41f0bec1441ad22eb257"
format_stamp: "Formatted at 2026-08-07 18:40:55 on dist-test-slave-2d19"
server_key: "c6ad38426f864d131fc73d22e370ed18"
server_key_iv: "4b731ae9e717eb1006e3fc75680cd691"
server_key_version: "encryptionkey@0"
I20260807 18:40:55.191253 28134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:55.222216 28134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:55.225598 28134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:55.225847 28134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:55.235294 28134 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.125:46065
I20260807 18:40:55.235519 28241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.125:46065 every 8 connection(s)
I20260807 18:40:55.236785 28134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
I20260807 18:40:55.239698 28126 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } attempt: 3
I20260807 18:40:55.240759 28242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:55.241765 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 28134
I20260807 18:40:55.241921 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-1/wal/instance
I20260807 18:40:55.242192 6697 external_mini_cluster.cc:1428] Setting key ec87126845ac673935ed1708c95ac732
I20260807 18:40:55.246029 28242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128054, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:40:55.253679 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.240320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:46065 (local address 127.0.0.1:33082)
0807 18:40:55.240621 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.240640 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:55.240750 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:55.243490 (+ 2740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:55.243497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:55.243734 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:55.243911 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.243922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.245927 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.245930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.246548 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.246558 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.247553 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.247556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.247672 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.248324 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:55.248339 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:55.249730 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:55.251821 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.251825 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.251827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.252039 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.252442 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.252446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.252448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.252541 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.253152 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:55.253155 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:55.253254 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 18:40:55.253377 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 18:40:55.253515 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260807 18:40:55.254626 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.241143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33082 (local address 127.6.138.125:46065)
0807 18:40:55.241608 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.241629 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.241675 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:40:55.242627 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.242662 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.243124 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.243534 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.244059 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.245785 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.246686 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.247433 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.247602 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.249925 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.249930 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.249940 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.249968 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.251692 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.252153 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.252160 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.252172 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.252345 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.252650 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.252653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.252655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.253038 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.253183 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.253525 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.253849 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for kudu/127.6.138.124@KRBTEST.COM; defaulting to no policy
I20260807 18:40:55.257030 28126 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:40:55.257766 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.257537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35532)
0807 18:40:55.257657 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.257705 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 18:40:55.258523 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.258225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35540)
0807 18:40:55.258395 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.258471 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260807 18:40:55.258661 28126 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:38933: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.124@KRBTEST.COM" created.
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:55.263870 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.248069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42433 (local address 127.0.0.1:36678)
0807 18:40:55.248538 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.248596 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:55.248775 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:55.249251 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:55.249261 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:55.249797 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:55.250059 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.250075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.251377 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.251384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.252699 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.252716 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.254638 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.254644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.254797 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.255374 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:55.255395 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:55.259970 (+ 4575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:55.262270 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.262280 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.262287 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.262516 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.262863 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.262865 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.262866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.262939 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.263367 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:55.263372 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:55.263476 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 18:40:55.263587 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 18:40:55.263729 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":145,"threads_started":1}
I20260807 18:40:55.264148 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.248119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.6.138.126:42433)
0807 18:40:55.248305 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.248311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.248676 (+ 365us) server_negotiation.cc:407] Connection header received
0807 18:40:55.248909 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.248916 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.248974 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.249035 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.250225 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.251185 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.252869 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.254344 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.254453 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.260129 (+ 5676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.260133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.260136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.260165 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.262142 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.262640 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.262646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.262651 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.262762 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.263047 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.263054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.263060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.263271 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.263357 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.263715 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.263888 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:40:55.265290 28242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:40:55.274678 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.265915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.6.138.125:46065)
0807 18:40:55.266064 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.266071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.266089 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:40:55.266155 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.266158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.266205 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.266295 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.266669 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.267647 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.268470 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.268911 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.269058 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.271224 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.271228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.271230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.271252 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.273000 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.273462 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.273466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.273468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.273558 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.273833 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.273837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.273838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.274105 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.274195 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.274452 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.274548 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:40:55.274693 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.265815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:46065 (local address 127.0.0.1:33094)
0807 18:40:55.265996 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.266007 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:55.266066 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:55.266314 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:55.266316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:55.266448 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:55.266535 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.266539 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.267763 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.267765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.268325 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.268350 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.269041 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.269044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.269129 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.269635 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:55.269645 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:55.271058 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:55.273164 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.273167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.273169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.273353 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.273645 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.273647 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.273649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.273719 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.274205 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:55.274207 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:55.274301 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 18:40:55.274393 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:40:55.274494 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":1}
I20260807 18:40:55.276129 28242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:40:55.276886 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.276629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35556)
0807 18:40:55.276755 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.276830 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 18:40:55.277098 28163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
I20260807 18:40:55.277973 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.277747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35566)
0807 18:40:55.277876 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.277926 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 18:40:55.278146 28242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:38933: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab.
Entry for principal kudu/127.6.138.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab.
I20260807 18:40:55.280481 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.124:38933
--webserver_interface=127.6.138.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab
--principal=kudu/127.6.138.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:42677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42433,127.6.138.125:46065,127.6.138.124:38933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:40:55.289813 28126 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } attempt: 1
I20260807 18:40:55.290652 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.290350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35576)
0807 18:40:55.290497 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.290565 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:40:55.291523 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.291311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35580)
0807 18:40:55.291426 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.291470 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:40:55.291715 28126 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:38933: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
I20260807 18:40:55.327445 28242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } attempt: 1
I20260807 18:40:55.328459 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.328109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35594)
0807 18:40:55.328315 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.328380 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 18:40:55.329200 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.328995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35610)
0807 18:40:55.329103 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.329145 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 18:40:55.329362 28242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:38933: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
I20260807 18:40:55.366905 28126 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } attempt: 2
I20260807 18:40:55.367959 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.367594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35622)
0807 18:40:55.367791 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.367864 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 18:40:55.368831 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.368627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35626)
0807 18:40:55.368736 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.368780 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:40:55.368968 28126 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:38933: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
I20260807 18:40:55.397528 28242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } attempt: 2
I20260807 18:40:55.398397 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.398081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35630)
0807 18:40:55.398249 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.398316 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:40:55.399179 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.398959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35642)
0807 18:40:55.399074 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.399126 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 18:40:55.399341 28242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:38933: Network error: Client connection negotiation failed: client connection to 127.6.138.124:38933: connect: Connection refused (error 111)
W20260807 18:40:55.403229 28253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:55.403659 28253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:55.403743 28253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:55.408043 28253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:40:55.408142 28253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:55.408200 28253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:55.408237 28253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:40:55.408300 28253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:40:55.413870 28253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:42677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42433,127.6.138.125:46065,127.6.138.124:38933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab
--principal=kudu/127.6.138.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.124:38933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.138.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:55.415129 28253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:55.416296 28253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:55.424840 28261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:55.424863 28258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:55.424994 28253 server_base.cc:1034] running on GCE node
W20260807 18:40:55.424858 28263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:55.425681 28253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:55.426929 28253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:55.428160 28253 hybrid_clock.cc:648] HybridClock initialized: now 1786128055428137 us; error 41 us; skew 500 ppm
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:55.431173 28253 init.cc:377] Logged in from keytab as kudu/127.6.138.124@KRBTEST.COM (short username kudu)
I20260807 18:40:55.432148 28253 webserver.cc:460] Webserver started at http://127.6.138.124:45649/ using document root <none> and password file <none>
I20260807 18:40:55.432754 28253 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:55.432819 28253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:40:55.433004 28253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:40:55.435062 28253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/data/instance:
uuid: "51f1ed79de024e0681f589ddc101e52b"
format_stamp: "Formatted at 2026-08-07 18:40:55 on dist-test-slave-2d19"
server_key: "58aa79c123e5a6ef09ba864cf987bcd8"
server_key_iv: "a87ca3acab45b648c9de9ee3c703710e"
server_key_version: "encryptionkey@0"
I20260807 18:40:55.435598 28253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/wal/instance:
uuid: "51f1ed79de024e0681f589ddc101e52b"
format_stamp: "Formatted at 2026-08-07 18:40:55 on dist-test-slave-2d19"
server_key: "58aa79c123e5a6ef09ba864cf987bcd8"
server_key_iv: "a87ca3acab45b648c9de9ee3c703710e"
server_key_version: "encryptionkey@0"
I20260807 18:40:55.439404 28253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:40:55.441879 28272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:55.443248 28253 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:40:55.443413 28253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/wal
uuid: "51f1ed79de024e0681f589ddc101e52b"
format_stamp: "Formatted at 2026-08-07 18:40:55 on dist-test-slave-2d19"
server_key: "58aa79c123e5a6ef09ba864cf987bcd8"
server_key_iv: "a87ca3acab45b648c9de9ee3c703710e"
server_key_version: "encryptionkey@0"
I20260807 18:40:55.443532 28253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:55.457402 28253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:55.460687 28253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:55.460888 28253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:55.467861 28253 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.124:38933
I20260807 18:40:55.467923 28351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.124:38933 every 8 connection(s)
I20260807 18:40:55.468135 28126 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } attempt: 3
I20260807 18:40:55.469146 28253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
I20260807 18:40:55.472199 28353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128054, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
I20260807 18:40:55.476392 28353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:40:55.478232 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 28253
I20260807 18:40:55.478375 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127881318083-6697-0/minicluster-data/master-2/wal/instance
I20260807 18:40:55.478693 6697 external_mini_cluster.cc:1428] Setting key 728053eb09cf8cc52390ac66d3ad96f2
I20260807 18:40:55.481771 28129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.468700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35644)
0807 18:40:55.468883 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.468902 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:55.469018 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:55.471489 (+ 2471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:55.471493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:55.471705 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:55.471821 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.471828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.473251 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.473257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.473898 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.473909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.474651 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.474657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.474741 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.475311 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:55.475323 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:55.476836 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:55.479606 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.479610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.479612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.479891 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.480372 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.480375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.480377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.480479 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.481241 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:55.481245 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:55.481343 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 18:40:55.481461 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 18:40:55.481599 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 18:40:55.482899 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.469498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.6.138.124:38933)
0807 18:40:55.469943 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.469962 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.470008 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:40:55.470712 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.470729 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.471193 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.471540 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.471999 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.473117 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.474046 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.474512 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.474690 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.477033 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.477039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.477051 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.477102 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.479437 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.480038 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.480043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.480048 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.480203 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.480588 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.480590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.480592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.481119 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.481271 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.481657 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.481969 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":301,"thread_start_us":135,"threads_started":1}
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
I20260807 18:40:55.487859 28126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23: Bootstrap starting.
I20260807 18:40:55.489241 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.479177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36690 (local address 127.6.138.126:42433)
0807 18:40:55.479312 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.479316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.479326 (+ 10us) server_negotiation.cc:407] Connection header received
0807 18:40:55.479357 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.479359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.479391 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.479491 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.480291 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.480994 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.482276 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.482552 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.482629 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.485838 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.485841 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.485843 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.485865 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.487229 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.487805 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.487808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.487810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.487893 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.488285 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.488289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.488291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.488533 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.488641 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.488962 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.489083 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:40:55.489275 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.478348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42433 (local address 127.0.0.1:36690)
0807 18:40:55.478774 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.478825 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:55.478933 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:55.479530 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:55.479537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:55.479933 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:55.480097 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.480105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.481112 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.481115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.482122 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.482135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.482779 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.482783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.482902 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.483471 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:55.483485 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:55.485684 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:55.487352 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.487361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.487366 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.487682 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.488028 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.488031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.488033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.488150 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.488651 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:55.488656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:55.488755 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 18:40:55.488904 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 18:40:55.489111 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":135,"threads_started":1}
I20260807 18:40:55.490289 28353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:40:55.490471 28242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } attempt: 3
I20260807 18:40:55.491786 28126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:55.491851 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.479715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.6.138.124:38933)
0807 18:40:55.480031 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.480041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.480177 (+ 136us) server_negotiation.cc:407] Connection header received
0807 18:40:55.480604 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.480608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.480651 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.480729 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.481668 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.482447 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.483909 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.484290 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.484412 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.487228 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.487231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.487234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.487259 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.489446 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.489955 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.489958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.489960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.490054 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.490342 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.490344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.490345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.490594 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.490790 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.491427 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.491530 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":105,"threads_started":1}
I20260807 18:40:55.492802 28126 log.cc:824] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:40:55.496635 28126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23: No bootstrap required, opened a new log
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:55.501849 28126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } }
I20260807 18:40:55.502126 28126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:55.502193 28126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b4b1285573545c8873e229acac66e23, State: Initialized, Role: FOLLOWER
I20260807 18:40:55.502224 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.491026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33108 (local address 127.6.138.125:46065)
0807 18:40:55.491423 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.491426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.491436 (+ 10us) server_negotiation.cc:407] Connection header received
0807 18:40:55.491465 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.491467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.491504 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.491580 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.492062 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.492786 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.493799 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.494155 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.494263 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.496962 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.496966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.496969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.496994 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.498912 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.500606 (+ 1694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.500610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.500612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.500728 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.501283 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.501287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.501288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.501575 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.501641 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.501990 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.502094 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":337}
I20260807 18:40:55.502568 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.491742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.6.138.124:38933)
0807 18:40:55.492403 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.492407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.492422 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:40:55.492457 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.492459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.492497 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.493130 (+ 633us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.493229 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.493712 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.495166 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.495488 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.495592 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.498004 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.498008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.498010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.498033 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.500201 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.500737 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.500740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.500742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.500824 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.501153 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.501157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.501160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.501393 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.501503 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.502297 (+ 794us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.502422 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:40:55.502780 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.490817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:46065 (local address 127.0.0.1:33108)
0807 18:40:55.490918 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.490933 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:55.491004 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:55.491617 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:55.491620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:55.491777 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:55.491907 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.491917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.492908 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.492912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.493672 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.493683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.494315 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.494322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.494447 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.495024 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:55.495039 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:55.496781 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:55.499532 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.499536 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.499539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.500499 (+ 960us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.500838 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.500843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.500846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.500930 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.501719 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:55.501724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:55.501807 (+ 83us) client_negotiation.cc:769] Sending connection context
0807 18:40:55.502419 (+ 612us) client_negotiation.cc:240] Negotiation successful
0807 18:40:55.502618 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 18:40:55.502753 28126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } }
I20260807 18:40:55.503345 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.490877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35662)
0807 18:40:55.491191 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.491205 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:55.491295 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:55.492642 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:55.492648 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:55.492843 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:55.492961 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.492969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.493864 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.493867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.495025 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.495036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.495709 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.495713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.495825 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.496301 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:55.496318 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:55.497864 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:55.500378 (+ 2514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.500382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.500385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.500633 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.500928 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.500931 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.500934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.501045 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.501552 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:55.501555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:55.501641 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 18:40:55.501745 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:40:55.501892 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":346}
I20260807 18:40:55.503381 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.503552 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:40:55.503993 28353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:40:55.504382 28126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:40:55.507031 28242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257: Bootstrap starting.
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128055, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
W20260807 18:40:55.510686 28386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:40:55.510778 28386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:40:55.509439 28020 master_runner.cc:186] Error getting master registration for 127.6.138.125:46065: OK, instance_id { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" instance_seqno: 1786128055232282 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:40:55.512862 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.504491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:38933 (local address 127.0.0.1:35672)
0807 18:40:55.504650 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:40:55.504663 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:40:55.504744 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:40:55.504934 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:40:55.504943 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:40:55.505104 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:40:55.505189 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.505197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.505994 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.506000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.506776 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:40:55.506787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.507380 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.507383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:40:55.507463 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.507930 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:40:55.507945 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:40:55.509430 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:40:55.511293 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.511300 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.511302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.511502 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.511788 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:40:55.511791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:40:55.511793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:40:55.511887 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:40:55.512327 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:40:55.512330 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:40:55.512412 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 18:40:55.512525 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:40:55.512666 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 18:40:55.512949 28020 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:40:55.512862 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.504557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.6.138.124:38933)
0807 18:40:55.504651 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.504656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.504706 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:40:55.504810 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.504816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.504857 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.504923 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.505319 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.505878 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.506907 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.507186 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.507282 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.509572 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.509575 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.509578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.509600 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.511180 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.511605 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.511610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.511612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.511691 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.512000 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.512006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.512008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.512177 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.512310 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.512570 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.512723 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":33}
I20260807 18:40:55.514966 28242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:55.515566 28353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b: Bootstrap starting.
I20260807 18:40:55.515906 28242 log.cc:824] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:55.516908 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.493408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.6.138.126:42433)
0807 18:40:55.493498 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.493502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.493643 (+ 141us) server_negotiation.cc:407] Connection header received
0807 18:40:55.493825 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.493828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.493867 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.493960 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.494733 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.495508 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.496591 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.496903 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.496986 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.499883 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.499886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.499889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.499915 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.501873 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.502394 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.502398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.502399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.502486 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.503025 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.503030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.503035 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.503263 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.503395 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.516590 (+ 13195us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.516743 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 18:40:55.518111 28353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:55.518148 28242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257: No bootstrap required, opened a new log
I20260807 18:40:55.518795 28353 log.cc:824] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:55.520637 28353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b: No bootstrap required, opened a new log
I20260807 18:40:55.523828 28242 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } }
I20260807 18:40:55.524053 28242 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:55.524114 28242 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e1c8c891be41f0bec1441ad22eb257, State: Initialized, Role: FOLLOWER
I20260807 18:40:55.524647 28242 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } }
I20260807 18:40:55.524926 28353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } }
I20260807 18:40:55.525213 28353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:55.525324 28353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51f1ed79de024e0681f589ddc101e52b, State: Initialized, Role: FOLLOWER
I20260807 18:40:55.526026 28242 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:40:55.526298 28353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } }
Aug 07 18:40:55 dist-test-slave-2d19 krb5kdc[28001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:40:55.527843 28353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:40:55.531257 28405 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:40:55.531332 28405 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:40:55.532756 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.521707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33124 (local address 127.6.138.125:46065)
0807 18:40:55.521847 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.521853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.521869 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:40:55.521920 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.521923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.521983 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.522087 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.522885 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.523664 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.524797 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.525129 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.525243 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.528537 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.528541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.528543 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.528569 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.530346 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.530842 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.530846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.530848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.531012 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.531346 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.531349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.531351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.531574 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.531773 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.532477 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.532601 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:40:55.536486 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:40:55.534767 28134 master_runner.cc:186] Error getting master registration for 127.6.138.124:38933: OK, instance_id { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" instance_seqno: 1786128055465966 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:40:55.538087 28134 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:40:55.544428 28391 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.544576 28391 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:40:55.545184 28392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.545286 28392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:40:55.546983 28253 master_runner.cc:263] No leader master found from master 51f1ed79de024e0681f589ddc101e52b
I20260807 18:40:55.547060 28253 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:40:55.548974 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.538611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.6.138.125:46065)
0807 18:40:55.538722 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.538726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.539726 (+ 1000us) server_negotiation.cc:407] Connection header received
0807 18:40:55.539910 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.539913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.539968 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.540038 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.541063 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.541736 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.542763 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.543086 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.543167 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.545256 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.545259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.545262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.545290 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.547090 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.547506 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.547509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.547511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.547601 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.547960 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.547963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.547965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.548185 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.548322 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.548729 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.548850 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 18:40:55.553874 28430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:40:55.553968 28430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:40:55.557682 28357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.539125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.6.138.124:38933)
0807 18:40:55.539253 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.539259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.549585 (+ 10326us) server_negotiation.cc:407] Connection header received
0807 18:40:55.549771 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.549774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.549821 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.549871 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.550745 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.551252 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.552048 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.552386 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.552499 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.553990 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.553995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.553999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.554029 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.555803 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.556321 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.556325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.556326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.556412 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.556712 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.556715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.556717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.556946 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.557062 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.557402 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.557526 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:40:55.563517 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:55.548662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.6.138.126:42433)
0807 18:40:55.548786 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:55.548790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:55.554552 (+ 5762us) server_negotiation.cc:407] Connection header received
0807 18:40:55.554616 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:55.554621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:55.554685 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:55.554772 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:55.555967 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.556752 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.558094 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:55.558396 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:55.558478 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:55.559997 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:55.560001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:55.560004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:55.560030 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:55.561483 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.562059 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.562063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.562065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.562159 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:55.562486 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:55.562490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:55.562492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:55.562718 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:55.562822 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:55.563224 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 18:40:55.563362 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:40:55.597976 28366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:40:55.598186 28366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } }
I20260807 18:40:55.599287 28366 leader_election.cc:288] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9e1c8c891be41f0bec1441ad22eb257 (127.6.138.125:46065), 51f1ed79de024e0681f589ddc101e52b (127.6.138.124:38933)
I20260807 18:40:55.600215 28309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b4b1285573545c8873e229acac66e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51f1ed79de024e0681f589ddc101e52b" is_pre_election: true
I20260807 18:40:55.600608 28309 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b4b1285573545c8873e229acac66e23 in term 0.
I20260807 18:40:55.601214 28205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b4b1285573545c8873e229acac66e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e1c8c891be41f0bec1441ad22eb257" is_pre_election: true
I20260807 18:40:55.601477 28205 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b4b1285573545c8873e229acac66e23 in term 0.
I20260807 18:40:55.602151 28042 leader_election.cc:302] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b4b1285573545c8873e229acac66e23, e9e1c8c891be41f0bec1441ad22eb257; no voters:
I20260807 18:40:55.604697 28366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:40:55.604795 28366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:40:55.604839 28366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:55.609732 28366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } }
I20260807 18:40:55.610280 28366 leader_election.cc:288] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [CANDIDATE]: Term 1 election: Requested vote from peers e9e1c8c891be41f0bec1441ad22eb257 (127.6.138.125:46065), 51f1ed79de024e0681f589ddc101e52b (127.6.138.124:38933)
I20260807 18:40:55.611044 28205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b4b1285573545c8873e229acac66e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e1c8c891be41f0bec1441ad22eb257"
I20260807 18:40:55.611186 28205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:55.617745 28309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b4b1285573545c8873e229acac66e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51f1ed79de024e0681f589ddc101e52b"
I20260807 18:40:55.617900 28309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:55.619302 28309 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b4b1285573545c8873e229acac66e23 in term 1.
I20260807 18:40:55.620313 28205 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b4b1285573545c8873e229acac66e23 in term 1.
I20260807 18:40:55.619802 28042 leader_election.cc:302] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b4b1285573545c8873e229acac66e23, 51f1ed79de024e0681f589ddc101e52b; no voters:
I20260807 18:40:55.623729 28366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:40:55.625567 28366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [term 1 LEADER]: Becoming Leader. State: Replica: 2b4b1285573545c8873e229acac66e23, State: Running, Role: LEADER
I20260807 18:40:55.626097 28366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } }
I20260807 18:40:55.628207 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b4b1285573545c8873e229acac66e23. Latest consensus state: current_term: 1 leader_uuid: "2b4b1285573545c8873e229acac66e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.635789 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [sys.catalog]: This master's current role is: LEADER
I20260807 18:40:55.636350 28437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:40:55.637506 28437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:40:55.643054 28309 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [term 1 FOLLOWER]: Refusing update from remote peer 2b4b1285573545c8873e229acac66e23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:40:55.643838 28435 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:40:55.650801 28392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b4b1285573545c8873e229acac66e23. Latest consensus state: current_term: 1 leader_uuid: "2b4b1285573545c8873e229acac66e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.650964 28392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:40:55.652231 28205 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [term 1 FOLLOWER]: Refusing update from remote peer 2b4b1285573545c8873e229acac66e23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:40:55.652959 28435 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:40:55.659377 28391 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b4b1285573545c8873e229acac66e23. Latest consensus state: current_term: 1 leader_uuid: "2b4b1285573545c8873e229acac66e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.659512 28391 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:40:55.667657 28435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b4b1285573545c8873e229acac66e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.667795 28435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [sys.catalog]: This master's current role is: LEADER
I20260807 18:40:55.669095 28392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b4b1285573545c8873e229acac66e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.669222 28392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:40:55.669703 28441 mvcc.cc:205] Tried to move back new op lower bound from 7315980515907141632 to 7315980515848593408. Current Snapshot: MvccSnapshot[applied={T|T < 7315980515907141632}]
I20260807 18:40:55.670776 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b4b1285573545c8873e229acac66e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.670888 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23 [sys.catalog]: This master's current role is: LEADER
I20260807 18:40:55.672165 28391 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b4b1285573545c8873e229acac66e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b4b1285573545c8873e229acac66e23" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42433 } } peers { permanent_uuid: "e9e1c8c891be41f0bec1441ad22eb257" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 46065 } } peers { permanent_uuid: "51f1ed79de024e0681f589ddc101e52b" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 38933 } } }
I20260807 18:40:55.672309 28391 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:40:55.673132 28444 mvcc.cc:205] Tried to move back new op lower bound from 7315980515907141632 to 7315980515848593408. Current Snapshot: MvccSnapshot[applied={T|T < 7315980515907141632}]
I20260807 18:40:55.673378 28442 mvcc.cc:205] Tried to move back new op lower bound from 7315980515907141632 to 7315980515848593408. Current Snapshot: MvccSnapshot[applied={T|T < 7315980515907141632}]
I20260807 18:40:55.683370 28437 catalog_manager.cc:1347] Generated new cluster ID: 8996c3465ae94c969b4d5a3cb8a5154a
I20260807 18:40:55.683458 28437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:40:55.711025 28437 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:40:55.713094 28437 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:40:55.747736 28437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b4b1285573545c8873e229acac66e23: Generated new TSK 0
I20260807 18:40:55.748881 28437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:40:56.532824 28405 catalog_manager.cc:1259] Loaded cluster ID: 8996c3465ae94c969b4d5a3cb8a5154a
I20260807 18:40:56.532949 28405 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257: loading cluster ID for follower catalog manager: success
I20260807 18:40:56.534649 28405 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257: acquiring CA information for follower catalog manager: success
I20260807 18:40:56.535815 28405 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e9e1c8c891be41f0bec1441ad22eb257: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:40:56.555240 28430 catalog_manager.cc:1259] Loaded cluster ID: 8996c3465ae94c969b4d5a3cb8a5154a
I20260807 18:40:56.555323 28430 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b: loading cluster ID for follower catalog manager: success
I20260807 18:40:56.556957 28430 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b: acquiring CA information for follower catalog manager: success
I20260807 18:40:56.558065 28430 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 51f1ed79de024e0681f589ddc101e52b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:40:56.816635 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 28020
I20260807 18:40:56.833894 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 28134
I20260807 18:40:56.847373 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 28253
2026-08-07T18:40:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3466 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:40:56 dist-test-slave-2d19 krb5kdc[28457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:40:56 dist-test-slave-2d19 krb5kdc[28457](info): set up 2 sockets
Aug 07 18:40:56 dist-test-slave-2d19 krb5kdc[28457](info): commencing operation
krb5kdc: starting...
W20260807 18:40:58.137984 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:40:58 dist-test-slave-2d19 krb5kdc[28457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:40:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:40:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:40:58.270761 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:36609
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:36609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:40:58.387188 28474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:40:58.387528 28474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:40:58.387614 28474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:40:58.391705 28474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:40:58.391816 28474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:40:58.391851 28474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:40:58.391911 28474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:40:58.391958 28474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:40:58.397550 28474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:36609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:36609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:40:58.398748 28474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:40:58.399816 28474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:40:58.409034 28484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:58.409195 28474 server_base.cc:1034] running on GCE node
W20260807 18:40:58.409046 28481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:40:58.409372 28482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:40:58.409941 28474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:40:58.411091 28474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:40:58.412372 28474 hybrid_clock.cc:648] HybridClock initialized: now 1786128058412345 us; error 46 us; skew 500 ppm
Aug 07 18:40:58 dist-test-slave-2d19 krb5kdc[28457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128058, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:40:58.416409 28474 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:40:58.417325 28474 webserver.cc:460] Webserver started at http://127.6.138.126:40309/ using document root <none> and password file <none>
I20260807 18:40:58.417922 28474 fs_manager.cc:359] Metadata directory not provided
I20260807 18:40:58.418026 28474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:40:58.418253 28474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:40:58.420413 28474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "e907cb414ab04b71a21fbe199bf6a1ad"
format_stamp: "Formatted at 2026-08-07 18:40:58 on dist-test-slave-2d19"
server_key: "0e807194ecf900c4dc7a85ce2b259696"
server_key_iv: "ed1aa2391b9af199c8fbb87ea93297e5"
server_key_version: "encryptionkey@0"
I20260807 18:40:58.421008 28474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "e907cb414ab04b71a21fbe199bf6a1ad"
format_stamp: "Formatted at 2026-08-07 18:40:58 on dist-test-slave-2d19"
server_key: "0e807194ecf900c4dc7a85ce2b259696"
server_key_iv: "ed1aa2391b9af199c8fbb87ea93297e5"
server_key_version: "encryptionkey@0"
I20260807 18:40:58.424888 28474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:40:58.427428 28498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:40:58.428802 28474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:40:58.428951 28474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "e907cb414ab04b71a21fbe199bf6a1ad"
format_stamp: "Formatted at 2026-08-07 18:40:58 on dist-test-slave-2d19"
server_key: "0e807194ecf900c4dc7a85ce2b259696"
server_key_iv: "ed1aa2391b9af199c8fbb87ea93297e5"
server_key_version: "encryptionkey@0"
I20260807 18:40:58.429088 28474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:40:58.441792 28474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:40:58.445328 28474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:40:58.445552 28474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:40:58.453626 28474 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:36609
I20260807 18:40:58.453648 28575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:36609 every 8 connection(s)
I20260807 18:40:58.454692 28474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:40:58.457654 28576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:40:58.458287 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 28474
I20260807 18:40:58.458412 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:40:58.458765 6697 external_mini_cluster.cc:1428] Setting key 24aa5bbec6d32aeef650afe4010fbcbc
I20260807 18:40:58.464066 28576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad: Bootstrap starting.
I20260807 18:40:58.466464 28576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad: Neither blocks nor log segments found. Creating new log.
I20260807 18:40:58.467247 28576 log.cc:824] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad: Log is configured to *not* fsync() on all Append() calls
I20260807 18:40:58.469368 28576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad: No bootstrap required, opened a new log
I20260807 18:40:58.473081 28576 raft_consensus.cc:348] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e907cb414ab04b71a21fbe199bf6a1ad" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36609 } }
Aug 07 18:40:58 dist-test-slave-2d19 krb5kdc[28457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:40:58.473462 28576 raft_consensus.cc:374] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:40:58.473583 28576 raft_consensus.cc:729] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e907cb414ab04b71a21fbe199bf6a1ad, State: Initialized, Role: FOLLOWER
I20260807 18:40:58.474187 28576 consensus_queue.cc:260] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e907cb414ab04b71a21fbe199bf6a1ad" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36609 } }
I20260807 18:40:58.474391 28576 raft_consensus.cc:388] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:40:58.474507 28576 raft_consensus.cc:482] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:40:58.474659 28576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:40:58.475687 28576 raft_consensus.cc:504] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e907cb414ab04b71a21fbe199bf6a1ad" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36609 } }
I20260807 18:40:58.476084 28576 leader_election.cc:302] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e907cb414ab04b71a21fbe199bf6a1ad; no voters:
I20260807 18:40:58.476550 28576 leader_election.cc:288] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:40:58.476655 28588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:40:58.476847 28588 raft_consensus.cc:686] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [term 1 LEADER]: Becoming Leader. State: Replica: e907cb414ab04b71a21fbe199bf6a1ad, State: Running, Role: LEADER
I20260807 18:40:58.477244 28588 consensus_queue.cc:237] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e907cb414ab04b71a21fbe199bf6a1ad" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36609 } }
I20260807 18:40:58.478063 28576 sys_catalog.cc:564] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:40:58.479202 28589 sys_catalog.cc:455] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e907cb414ab04b71a21fbe199bf6a1ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e907cb414ab04b71a21fbe199bf6a1ad" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36609 } } }
I20260807 18:40:58.479372 28589 sys_catalog.cc:458] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [sys.catalog]: This master's current role is: LEADER
I20260807 18:40:58.479799 28602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:40:58.479754 28590 sys_catalog.cc:455] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader e907cb414ab04b71a21fbe199bf6a1ad. Latest consensus state: current_term: 1 leader_uuid: "e907cb414ab04b71a21fbe199bf6a1ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e907cb414ab04b71a21fbe199bf6a1ad" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 36609 } } }
I20260807 18:40:58.479909 28590 sys_catalog.cc:458] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad [sys.catalog]: This master's current role is: LEADER
I20260807 18:40:58.480475 28581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:40:58.460321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.6.138.126:36609)
0807 18:40:58.460799 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 18:40:58.460815 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:40:58.460852 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:40:58.461546 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:40:58.461565 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:40:58.461915 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:40:58.462274 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:40:58.463146 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:58.464141 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:58.465088 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:40:58.465580 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:40:58.465757 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:40:58.474337 (+ 8580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:40:58.474343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:40:58.474358 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:40:58.474413 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:40:58.476564 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:58.477207 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:58.477213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:58.477218 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:58.477386 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:40:58.477702 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:40:58.477706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:40:58.477708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:40:58.478184 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:40:58.478437 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 18:40:58.478769 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 18:40:58.479115 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":164,"threads_started":1}
I20260807 18:40:58.483544 28602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:40:58.484184 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:40:58.489446 28602 catalog_manager.cc:1347] Generated new cluster ID: 96f17997b30246cfbfaa8cc59161be65
I20260807 18:40:58.489535 28602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:40:58.508737 28602 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:40:58.509745 28602 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:40:58.519430 28602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e907cb414ab04b71a21fbe199bf6a1ad: Generated new TSK 0
I20260807 18:40:58.520195 28602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:40:58.541163 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 28474
2026-08-07T18:40:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1694 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:40:58 dist-test-slave-2d19 krb5kdc[28620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:40:58 dist-test-slave-2d19 krb5kdc[28620](info): set up 2 sockets
Aug 07 18:40:58 dist-test-slave-2d19 krb5kdc[28620](info): commencing operation
krb5kdc: starting...
W20260807 18:40:59.828984 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:40:59 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:40:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:40:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:40:59.968063 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37861
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:37861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:00.091116 28640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:00.091477 28640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:00.091566 28640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:00.095809 28640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:41:00.095903 28640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:00.095938 28640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:00.095980 28640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:41:00.096017 28640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:41:00.101514 28640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:37861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:00.102732 28640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:00.103835 28640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:00.113512 28647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:00.113514 28650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:00.113514 28646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:00.113608 28640 server_base.cc:1034] running on GCE node
I20260807 18:41:00.114514 28640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:00.115617 28640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:00.116791 28640 hybrid_clock.cc:648] HybridClock initialized: now 1786128060116756 us; error 48 us; skew 500 ppm
Aug 07 18:41:00 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128060, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:00.119738 28640 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:41:00.120760 28640 webserver.cc:460] Webserver started at http://127.6.138.126:40109/ using document root <none> and password file <none>
I20260807 18:41:00.121341 28640 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:00.121404 28640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:00.121680 28640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:41:00.123816 28640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "9fcf9b79cf844a3a980bbd218a63dfcc"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "74d63a14f6e1c5806b4932180d5ad0f0"
server_key_iv: "5bd9b3c5caf8a47602738e677f0da31d"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.124480 28640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "9fcf9b79cf844a3a980bbd218a63dfcc"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "74d63a14f6e1c5806b4932180d5ad0f0"
server_key_iv: "5bd9b3c5caf8a47602738e677f0da31d"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.128379 28640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:41:00.130874 28663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.132328 28640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:00.132474 28640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "9fcf9b79cf844a3a980bbd218a63dfcc"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "74d63a14f6e1c5806b4932180d5ad0f0"
server_key_iv: "5bd9b3c5caf8a47602738e677f0da31d"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.132602 28640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:00.153345 28640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:00.157212 28640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:00.157444 28640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:00.164620 28640 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:37861
I20260807 18:41:00.164928 28756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:37861 every 8 connection(s)
I20260807 18:41:00.165652 28640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:41:00.165935 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 28640
I20260807 18:41:00.166076 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:41:00.166430 6697 external_mini_cluster.cc:1428] Setting key 5efc103edccbefaa416318322770fada
I20260807 18:41:00.169097 28757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:41:00 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:00.176491 28757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Bootstrap starting.
I20260807 18:41:00.178862 28757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:00.179822 28757 log.cc:824] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:00.181878 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.167866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.6.138.126:37861)
0807 18:41:00.168322 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:00.168342 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:00.168386 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:41:00.169087 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:00.169105 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:00.169491 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:00.169774 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:00.170699 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.171730 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.172933 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.173399 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.173518 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.176777 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:00.176782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:00.176796 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:00.176823 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:00.178626 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.179152 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.179157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.179162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.179276 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.179584 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.179587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.179588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.180029 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:00.180175 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:00.180596 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 18:41:00.180914 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":142,"threads_started":1}
I20260807 18:41:00.182238 28757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: No bootstrap required, opened a new log
I20260807 18:41:00.186213 28757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:00.186465 28757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:00.186501 28757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fcf9b79cf844a3a980bbd218a63dfcc, State: Initialized, Role: FOLLOWER
I20260807 18:41:00.186976 28757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:00.187116 28757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:41:00.187203 28757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:41:00.187310 28757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:00.188405 28757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:00.188768 28757 leader_election.cc:302] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fcf9b79cf844a3a980bbd218a63dfcc; no voters:
I20260807 18:41:00.189131 28757 leader_election.cc:288] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:41:00.189286 28764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:41:00.189548 28764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 1 LEADER]: Becoming Leader. State: Replica: 9fcf9b79cf844a3a980bbd218a63dfcc, State: Running, Role: LEADER
I20260807 18:41:00.189955 28764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:00.190449 28757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:41:00.191797 28765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } } }
I20260807 18:41:00.191948 28765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:00.192447 28766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fcf9b79cf844a3a980bbd218a63dfcc. Latest consensus state: current_term: 1 leader_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } } }
I20260807 18:41:00.192523 28778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:41:00.192586 28766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:00.195484 28778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:41:00.200803 28778 catalog_manager.cc:1347] Generated new cluster ID: ea6b0d511d5549da942d24999b2ade8b
I20260807 18:41:00.200877 28778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:41:00.214212 28778 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:41:00.215508 28778 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:41:00.222558 28778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Generated new TSK 0
I20260807 18:41:00.223379 28778 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:41:00.263778 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37861
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:00.383025 28796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:00.383322 28796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:00.383368 28796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:00.387609 28796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:00.387686 28796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:00.387748 28796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:41:00.393636 28796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:00.394735 28796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:00.395789 28796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:00.406064 28804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:00.406299 28796 server_base.cc:1034] running on GCE node
W20260807 18:41:00.406069 28805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:00.406069 28807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:00.406867 28796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:00.407503 28796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:00.408710 28796 hybrid_clock.cc:648] HybridClock initialized: now 1786128060408677 us; error 51 us; skew 500 ppm
Aug 07 18:41:00 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128060, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:00.411603 28796 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:41:00.412618 28796 webserver.cc:460] Webserver started at http://127.6.138.65:36403/ using document root <none> and password file <none>
I20260807 18:41:00.413209 28796 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:00.413305 28796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:00.413560 28796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:41:00.415660 28796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "d6a3df0b65c64797b1d1d1d89cf996a2"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "b6a3d518014b4a2833da859e6d4830b7"
server_key_iv: "f2cf48bae1cff0a529a3d60bc0ff7474"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.416245 28796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "d6a3df0b65c64797b1d1d1d89cf996a2"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "b6a3d518014b4a2833da859e6d4830b7"
server_key_iv: "f2cf48bae1cff0a529a3d60bc0ff7474"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.420372 28796 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:41:00.423812 28817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.425180 28796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:00.425320 28796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "d6a3df0b65c64797b1d1d1d89cf996a2"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "b6a3d518014b4a2833da859e6d4830b7"
server_key_iv: "f2cf48bae1cff0a529a3d60bc0ff7474"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.425473 28796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:00.441908 28796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:00.445528 28796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:00.445752 28796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:00.446422 28796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:00.447299 28796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:00.447348 28796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.447430 28796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:00.447474 28796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.457049 28796 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:44771
I20260807 18:41:00.457072 28975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:44771 every 8 connection(s)
I20260807 18:41:00.457989 28796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:41:00.461658 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 28796
I20260807 18:41:00.461828 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:41:00.462152 6697 external_mini_cluster.cc:1428] Setting key 9c89ff322b61600219f0afb447621a9d
Aug 07 18:41:00 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128060, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:41:00.474314 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.459957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:35557 (local address 127.6.138.126:37861)
0807 18:41:00.460171 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:00.460175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:00.461179 (+ 1004us) server_negotiation.cc:407] Connection header received
0807 18:41:00.462826 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:00.462829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:00.462884 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:00.462986 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:00.464612 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.465148 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.466720 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.467138 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.467259 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.470371 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:00.470374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:00.470376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:00.470401 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:00.471960 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.472648 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.472651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.472653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.472732 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.473168 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.473170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.473171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.473394 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:00.473494 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:00.474063 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 18:41:00.474183 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:41:00.475221 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.460533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:35557)
0807 18:41:00.460995 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:00.461038 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:00.462620 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:00.463188 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:00.463197 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:00.463700 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:00.464437 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.464463 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.465278 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.465284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.466490 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.466501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.467302 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.467305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.467503 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.468183 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:00.468209 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:00.470168 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:00.472093 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:00.472100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:00.472117 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:00.472526 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:00.472882 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:00.472885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:00.472888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:00.473062 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:00.473534 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:00.473539 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:00.473785 (+ 246us) client_negotiation.cc:769] Sending connection context
0807 18:41:00.473951 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 18:41:00.474226 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
I20260807 18:41:00.476744 28976 heartbeater.cc:347] Connected to a master server at 127.6.138.126:37861
I20260807 18:41:00.477026 28976 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:00.477567 28976 heartbeater.cc:511] Master 127.6.138.126:37861 requested a full tablet report, sending...
I20260807 18:41:00.479162 28691 ts_manager.cc:194] Registered new tserver with Master: d6a3df0b65c64797b1d1d1d89cf996a2 (127.6.138.65:44771)
I20260807 18:41:00.480412 28691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:35557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:41:00.494908 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37861
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:00.612457 28985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:00.612740 28985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:00.612787 28985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:00.617602 28985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:00.617678 28985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:00.617740 28985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:41:00.623296 28985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:00.624450 28985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:00.625499 28985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:00.635865 28996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:00.636106 28998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:00.636169 28985 server_base.cc:1034] running on GCE node
W20260807 18:41:00.636304 29001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:00.636996 28985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:00.637965 28985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:00.639135 28985 hybrid_clock.cc:648] HybridClock initialized: now 1786128060639107 us; error 46 us; skew 500 ppm
Aug 07 18:41:00 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128060, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:00.642758 28985 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:41:00.645267 28985 webserver.cc:460] Webserver started at http://127.6.138.66:34633/ using document root <none> and password file <none>
I20260807 18:41:00.645871 28985 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:00.645965 28985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:00.646222 28985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:41:00.648370 28985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "b0fb355706ad47a7ba33269b6824378d"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "1e6a5d4228d7a1764c435388cb5c9ce7"
server_key_iv: "30d3804db41e586c406a6fe692d77cb0"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.649030 28985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "b0fb355706ad47a7ba33269b6824378d"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "1e6a5d4228d7a1764c435388cb5c9ce7"
server_key_iv: "30d3804db41e586c406a6fe692d77cb0"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.653445 28985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:41:00.655941 29010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.657248 28985 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:41:00.657392 28985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "b0fb355706ad47a7ba33269b6824378d"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "1e6a5d4228d7a1764c435388cb5c9ce7"
server_key_iv: "30d3804db41e586c406a6fe692d77cb0"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.657508 28985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:00.679325 28985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:00.683086 28985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:00.683305 28985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:00.683941 28985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:00.684885 28985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:00.684959 28985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.685041 28985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:00.685093 28985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.694895 28985 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:34457
I20260807 18:41:00.694945 29184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:34457 every 8 connection(s)
I20260807 18:41:00.695910 28985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:41:00.703406 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 28985
I20260807 18:41:00.703522 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:41:00.703800 6697 external_mini_cluster.cc:1428] Setting key 3440776802fd8b5c666979a2e176b6cd
Aug 07 18:41:00 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128060, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:00.710420 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.697882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:52285 (local address 127.6.138.126:37861)
0807 18:41:00.698065 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:00.698068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:00.698910 (+ 842us) server_negotiation.cc:407] Connection header received
0807 18:41:00.700288 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:00.700291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:00.700349 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:00.700465 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:00.701637 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.702176 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.703431 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.703727 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.703815 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.706653 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:00.706656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:00.706658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:00.706681 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:00.708160 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.708724 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.708728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.708730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.708834 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.709284 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.709287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.709288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.709467 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:00.709594 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:00.710194 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 18:41:00.710299 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:41:00.711722 29188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.698284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:52285)
0807 18:41:00.698754 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:00.698797 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:00.700047 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:00.700652 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:00.700658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:00.701008 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:00.701476 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.701490 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.702311 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.702320 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.703273 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.703288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.703895 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.703900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.704101 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.704820 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:00.704841 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:00.706463 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:00.708320 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:00.708326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:00.708342 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:00.708615 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:00.708955 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:00.708961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:00.708966 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:00.709172 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:00.709578 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:00.709587 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:00.709919 (+ 332us) client_negotiation.cc:769] Sending connection context
0807 18:41:00.710149 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 18:41:00.710556 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
I20260807 18:41:00.713066 29186 heartbeater.cc:347] Connected to a master server at 127.6.138.126:37861
I20260807 18:41:00.713325 29186 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:00.713837 29186 heartbeater.cc:511] Master 127.6.138.126:37861 requested a full tablet report, sending...
I20260807 18:41:00.714985 28691 ts_manager.cc:194] Registered new tserver with Master: b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457)
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:41:00.715667 28691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:52285
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:41:00.735299 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37861
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:00.854043 29193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:00.854308 29193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:00.854352 29193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:00.858316 29193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:00.858381 29193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:00.858438 29193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:41:00.863749 29193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:00.864921 29193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:00.865965 29193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:41:00.876185 29193 server_base.cc:1034] running on GCE node
W20260807 18:41:00.876160 29202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:00.876161 29205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:00.876452 29203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:00.876942 29193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:00.877724 29193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:00.878906 29193 hybrid_clock.cc:648] HybridClock initialized: now 1786128060878885 us; error 43 us; skew 500 ppm
Aug 07 18:41:00 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128060, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:00.882085 29193 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:41:00.883155 29193 webserver.cc:460] Webserver started at http://127.6.138.67:33789/ using document root <none> and password file <none>
I20260807 18:41:00.883797 29193 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:00.883895 29193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:00.884160 29193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:41:00.886613 29193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "d5f36c29f6f2487ba435aac2aea35602"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "98a9789362fed2da9e0defdb5bb70bd5"
server_key_iv: "885370df6d8c454de0692be191b2d81b"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.887257 29193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "d5f36c29f6f2487ba435aac2aea35602"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "98a9789362fed2da9e0defdb5bb70bd5"
server_key_iv: "885370df6d8c454de0692be191b2d81b"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.891664 29193 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 18:41:00.894546 29217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.895864 29193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:41:00.896005 29193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "d5f36c29f6f2487ba435aac2aea35602"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "98a9789362fed2da9e0defdb5bb70bd5"
server_key_iv: "885370df6d8c454de0692be191b2d81b"
server_key_version: "encryptionkey@0"
I20260807 18:41:00.896160 29193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:00.913803 29193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:00.917207 29193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:00.917409 29193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:00.918026 29193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:00.918862 29193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:00.918933 29193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.919000 29193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:00.919052 29193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:00.929117 29193 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:44513
I20260807 18:41:00.929173 29380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:44513 every 8 connection(s)
I20260807 18:41:00.930104 29193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:41:00.933468 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 29193
I20260807 18:41:00.933584 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:41:00.933919 6697 external_mini_cluster.cc:1428] Setting key b28352b948d4f8f0b427c5f1719d21ff
Aug 07 18:41:00 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128060, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:00.945015 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.932117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:56127 (local address 127.6.138.126:37861)
0807 18:41:00.932355 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:00.932361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:00.933253 (+ 892us) server_negotiation.cc:407] Connection header received
0807 18:41:00.934898 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:00.934901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:00.934957 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:00.935077 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:00.936704 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.937240 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.938433 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.938671 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.938787 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.941318 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:00.941322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:00.941324 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:00.941347 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:00.942664 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.943231 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.943235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.943236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.943318 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.943697 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.943700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.943702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.943900 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:00.944012 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:00.944640 (+ 628us) server_negotiation.cc:299] Negotiation successful
0807 18:41:00.944771 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:41:00.945837 29384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.932582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:56127)
0807 18:41:00.933077 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:00.933124 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:00.934684 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:00.935272 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:00.935279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:00.935772 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:00.936549 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.936586 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.937370 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.937376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.938298 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.938307 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.938836 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.938841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.939030 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.939593 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:00.939611 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:00.941139 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:00.942803 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:00.942809 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:00.942826 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:00.943104 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:00.943432 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:00.943435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:00.943437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:00.943589 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:00.944012 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:00.944020 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:00.944361 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 18:41:00.944577 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 18:41:00.944929 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":116,"threads_started":1}
I20260807 18:41:00.947296 29381 heartbeater.cc:347] Connected to a master server at 127.6.138.126:37861
I20260807 18:41:00.947584 29381 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:00.948170 29381 heartbeater.cc:511] Master 127.6.138.126:37861 requested a full tablet report, sending...
I20260807 18:41:00.949307 28691 ts_manager.cc:194] Registered new tserver with Master: d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513)
I20260807 18:41:00.949903 28691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:56127
I20260807 18:41:00.960505 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:41:00.969833 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.962670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47504 (local address 127.6.138.126:37861)
0807 18:41:00.962810 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:00.962813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:00.962982 (+ 169us) server_negotiation.cc:407] Connection header received
0807 18:41:00.963082 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:00.963084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:00.963126 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:00.963190 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:00.963853 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.964439 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.965255 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.965466 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.965562 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.966868 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:00.966871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:00.966873 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:00.966897 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:00.968448 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.968806 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.968808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.968809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.968883 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:00.969107 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:00.969109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:00.969110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:00.969319 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:00.969420 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:00.969620 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 18:41:00.969713 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:41:00.972820 28691 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:41:00.976002 28691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:41:00.990640 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.985130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.6.138.66:34457)
0807 18:41:00.986097 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:00.986113 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:00.986132 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:41:00.986194 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:00.986197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:00.986329 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:00.986486 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:00.986708 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.987443 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.989138 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.989868 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.990006 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.990175 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:00.990359 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 18:41:00.990478 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":164,"threads_started":1}
I20260807 18:41:00.990653 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.984370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:44513 (local address 127.0.0.1:54826)
0807 18:41:00.984897 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:00.984944 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:00.985089 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:00.986046 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:00.986053 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:00.986079 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:00.986229 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.986236 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.987903 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.987906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.989096 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.989106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.990101 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.990105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.990209 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.990236 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:41:00.990318 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 18:41:00.990525 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":49,"thread_start_us":92,"threads_started":1}
I20260807 18:41:00.990834 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.985136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:44771 (local address 127.0.0.1:59780)
0807 18:41:00.985473 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:00.985488 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:00.985567 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:00.986649 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:00.986652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:00.986670 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:00.986759 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.986765 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.988219 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.988223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.989409 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.989420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.990460 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.990467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.990550 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.990573 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:41:00.990624 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 18:41:00.990693 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":48,"thread_start_us":96,"threads_started":1}
I20260807 18:41:00.990954 29395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.984571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:34457 (local address 127.0.0.1:57450)
0807 18:41:00.985126 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:00.985139 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:00.985232 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:00.986463 (+ 1231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:00.986465 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:00.986483 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:00.986604 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.986613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.987586 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.987591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.988992 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:00.989004 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.990019 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.990023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:00.990136 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.990175 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:41:00.990328 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 18:41:00.990852 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":59,"thread_start_us":105,"threads_started":1}
I20260807 18:41:00.990983 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.984561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.6.138.67:44513)
0807 18:41:00.985491 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:00.985505 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:00.985530 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:41:00.985611 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:00.985617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:00.985832 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:00.986387 (+ 555us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:00.986558 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.987758 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.989217 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.989954 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.990516 (+ 562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.990680 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:00.990729 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 18:41:00.990814 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":161,"threads_started":1}
I20260807 18:41:00.991614 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:00.985688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59780 (local address 127.6.138.65:44771)
0807 18:41:00.986167 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:00.986174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:00.986196 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:41:00.986272 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:00.986279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:00.986479 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:00.986908 (+ 429us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:00.987039 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.988083 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.989539 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:00.990330 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:00.991138 (+ 808us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:00.991236 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:00.991309 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 18:41:00.991452 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":128,"threads_started":1}
I20260807 18:41:00.994203 29288 tablet_service.cc:1467] Processing CreateTablet for tablet 7e96ad527c6f428a86e8803d62e2cfbc (DEFAULT_TABLE table=test-table [id=60ba9afc7ef44840a3dc44d39a419b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:00.994361 28885 tablet_service.cc:1467] Processing CreateTablet for tablet 7e96ad527c6f428a86e8803d62e2cfbc (DEFAULT_TABLE table=test-table [id=60ba9afc7ef44840a3dc44d39a419b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:00.994361 29081 tablet_service.cc:1467] Processing CreateTablet for tablet 7e96ad527c6f428a86e8803d62e2cfbc (DEFAULT_TABLE table=test-table [id=60ba9afc7ef44840a3dc44d39a419b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:00.995265 29288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e96ad527c6f428a86e8803d62e2cfbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:00.995380 29081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e96ad527c6f428a86e8803d62e2cfbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:00.995380 28885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e96ad527c6f428a86e8803d62e2cfbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:01.001726 29405 tablet_bootstrap.cc:492] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: Bootstrap starting.
I20260807 18:41:01.003254 29406 tablet_bootstrap.cc:492] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Bootstrap starting.
I20260807 18:41:01.004431 29405 tablet_bootstrap.cc:654] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:01.004617 29407 tablet_bootstrap.cc:492] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: Bootstrap starting.
I20260807 18:41:01.005270 29405 log.cc:824] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:01.005379 29406 tablet_bootstrap.cc:654] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:01.006225 29406 log.cc:824] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:01.007230 29407 tablet_bootstrap.cc:654] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:01.007541 29405 tablet_bootstrap.cc:492] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: No bootstrap required, opened a new log
I20260807 18:41:01.007761 29405 ts_tablet_manager.cc:1397] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:41:01.007999 29407 log.cc:824] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:01.008322 29406 tablet_bootstrap.cc:492] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: No bootstrap required, opened a new log
I20260807 18:41:01.008514 29406 ts_tablet_manager.cc:1397] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260807 18:41:01.009737 29407 tablet_bootstrap.cc:492] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: No bootstrap required, opened a new log
I20260807 18:41:01.009944 29407 ts_tablet_manager.cc:1397] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:41:01.011942 29405 raft_consensus.cc:348] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.012246 29405 raft_consensus.cc:374] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:01.012334 29405 raft_consensus.cc:729] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f36c29f6f2487ba435aac2aea35602, State: Initialized, Role: FOLLOWER
I20260807 18:41:01.012807 29405 consensus_queue.cc:260] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.012873 29407 raft_consensus.cc:348] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.013142 29407 raft_consensus.cc:374] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:01.013003 29406 raft_consensus.cc:348] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.013201 29407 raft_consensus.cc:729] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fb355706ad47a7ba33269b6824378d, State: Initialized, Role: FOLLOWER
I20260807 18:41:01.013264 29406 raft_consensus.cc:374] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:01.013314 29406 raft_consensus.cc:729] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a3df0b65c64797b1d1d1d89cf996a2, State: Initialized, Role: FOLLOWER
I20260807 18:41:01.013665 29407 consensus_queue.cc:260] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.013706 29406 consensus_queue.cc:260] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.013870 29405 ts_tablet_manager.cc:1428] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:41:01.014344 29381 heartbeater.cc:503] Master 127.6.138.126:37861 was elected leader, sending a full tablet report...
I20260807 18:41:01.014613 29407 ts_tablet_manager.cc:1428] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:41:01.014616 28976 heartbeater.cc:503] Master 127.6.138.126:37861 was elected leader, sending a full tablet report...
I20260807 18:41:01.014642 29406 ts_tablet_manager.cc:1428] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:41:01.015132 29186 heartbeater.cc:503] Master 127.6.138.126:37861 was elected leader, sending a full tablet report...
I20260807 18:41:01.076138 29413 raft_consensus.cc:482] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:41:01.076431 29413 raft_consensus.cc:504] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.077872 29413 leader_election.cc:288] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457), d6a3df0b65c64797b1d1d1d89cf996a2 (127.6.138.65:44771)
I20260807 18:41:01.082924 29384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.078209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:34457 (local address 127.6.138.67:49193)
0807 18:41:01.078375 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:01.078392 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:01.078499 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:01.078989 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:01.078992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:01.079014 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:01.079157 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.079196 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.080160 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.080164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.081584 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.081597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.082456 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.082459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.082569 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.082594 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:41:01.082694 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:41:01.082773 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":28}
I20260807 18:41:01.082924 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.078500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:49193 (local address 127.6.138.66:34457)
0807 18:41:01.078681 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:01.078689 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:01.078712 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:41:01.078785 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:01.078790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:01.078865 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:01.078971 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:01.079331 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.080061 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.081736 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.082301 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.082449 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.082507 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:01.082711 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 18:41:01.082788 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:41:01.083511 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.079407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:59633 (local address 127.6.138.65:44771)
0807 18:41:01.079551 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:01.079556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:01.079763 (+ 207us) server_negotiation.cc:407] Connection header received
0807 18:41:01.079869 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:01.079872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:01.079916 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:01.079992 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:01.080398 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.081090 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.082443 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.082873 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.083008 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.083058 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:01.083294 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 18:41:01.083390 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:41:01.083608 29108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e96ad527c6f428a86e8803d62e2cfbc" candidate_uuid: "d5f36c29f6f2487ba435aac2aea35602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fb355706ad47a7ba33269b6824378d" is_pre_election: true
I20260807 18:41:01.083952 29418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.079245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:44771 (local address 127.6.138.67:59633)
0807 18:41:01.079656 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:01.079673 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:01.079783 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:01.080021 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:01.080027 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:01.080050 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:01.080275 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.080289 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.081217 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.081221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.082325 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.082338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.083020 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.083024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.083129 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.083163 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:41:01.083694 (+ 531us) client_negotiation.cc:240] Negotiation successful
0807 18:41:01.083777 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":23,"thread_start_us":161,"threads_started":1}
I20260807 18:41:01.083976 29108 raft_consensus.cc:2454] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5f36c29f6f2487ba435aac2aea35602 in term 0.
I20260807 18:41:01.084609 29220 leader_election.cc:302] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0fb355706ad47a7ba33269b6824378d, d5f36c29f6f2487ba435aac2aea35602; no voters:
I20260807 18:41:01.084793 28913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e96ad527c6f428a86e8803d62e2cfbc" candidate_uuid: "d5f36c29f6f2487ba435aac2aea35602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" is_pre_election: true
I20260807 18:41:01.084976 29413 raft_consensus.cc:2781] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:41:01.085078 29413 raft_consensus.cc:482] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:41:01.085139 29413 raft_consensus.cc:3037] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:01.085111 28913 raft_consensus.cc:2454] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5f36c29f6f2487ba435aac2aea35602 in term 0.
I20260807 18:41:01.086788 29413 raft_consensus.cc:504] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.087248 29413 leader_election.cc:288] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [CANDIDATE]: Term 1 election: Requested vote from peers b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457), d6a3df0b65c64797b1d1d1d89cf996a2 (127.6.138.65:44771)
I20260807 18:41:01.087618 29108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e96ad527c6f428a86e8803d62e2cfbc" candidate_uuid: "d5f36c29f6f2487ba435aac2aea35602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fb355706ad47a7ba33269b6824378d"
I20260807 18:41:01.087635 28913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e96ad527c6f428a86e8803d62e2cfbc" candidate_uuid: "d5f36c29f6f2487ba435aac2aea35602" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2"
I20260807 18:41:01.087770 29108 raft_consensus.cc:3037] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:01.087795 28913 raft_consensus.cc:3037] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:01.089421 28913 raft_consensus.cc:2454] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5f36c29f6f2487ba435aac2aea35602 in term 1.
I20260807 18:41:01.089466 29108 raft_consensus.cc:2454] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5f36c29f6f2487ba435aac2aea35602 in term 1.
I20260807 18:41:01.089779 29220 leader_election.cc:302] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b0fb355706ad47a7ba33269b6824378d, d5f36c29f6f2487ba435aac2aea35602, d6a3df0b65c64797b1d1d1d89cf996a2; no voters:
I20260807 18:41:01.089993 29413 raft_consensus.cc:2781] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:41:01.090268 29413 raft_consensus.cc:686] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 1 LEADER]: Becoming Leader. State: Replica: d5f36c29f6f2487ba435aac2aea35602, State: Running, Role: LEADER
I20260807 18:41:01.090751 29413 consensus_queue.cc:237] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.096315 28691 catalog_manager.cc:5495] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "d5f36c29f6f2487ba435aac2aea35602" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } health_report { overall_health: UNKNOWN } } }
I20260807 18:41:01.119635 29396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.114284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54828 (local address 127.6.138.67:44513)
0807 18:41:01.114459 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:01.114463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:01.114481 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:41:01.114639 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:01.114642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:01.114727 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:01.114839 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:41:01.115189 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.116048 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.117526 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.117917 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.118061 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.118467 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:41:01.119172 (+ 705us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:41:01.119299 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:01.119398 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 18:41:01.119468 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:41:01.127978 29108 raft_consensus.cc:1261] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [term 1 FOLLOWER]: Refusing update from remote peer d5f36c29f6f2487ba435aac2aea35602: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:41:01.128060 28913 raft_consensus.cc:1261] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Refusing update from remote peer d5f36c29f6f2487ba435aac2aea35602: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:41:01.128841 29413 consensus_queue.cc:1035] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:41:01.129043 29419 consensus_queue.cc:1035] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:41:01.139832 29423 mvcc.cc:205] Tried to move back new op lower bound from 7315980538371031040 to 7315980538231357440. Current Snapshot: MvccSnapshot[applied={T|T < 7315980538371031040}]
I20260807 18:41:01.147553 28690 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:41:01.147754 28690 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:41:01.148548 29424 mvcc.cc:205] Tried to move back new op lower bound from 7315980538371031040 to 7315980538231357440. Current Snapshot: MvccSnapshot[applied={T|T < 7315980538371031040}]
I20260807 18:41:01.150763 28690 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Sending DeleteTablet for 3 replicas of tablet 7e96ad527c6f428a86e8803d62e2cfbc
I20260807 18:41:01.151495 29288 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e96ad527c6f428a86e8803d62e2cfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:01 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:54826
I20260807 18:41:01.152516 29081 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e96ad527c6f428a86e8803d62e2cfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:01 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:57450
I20260807 18:41:01.153014 29440 tablet_replica.cc:330] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: stopping tablet replica
I20260807 18:41:01.153354 29440 raft_consensus.cc:2229] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:41:01.153407 28885 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e96ad527c6f428a86e8803d62e2cfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:01 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:59780
I20260807 18:41:01.153604 29439 tablet_replica.cc:330] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: stopping tablet replica
I20260807 18:41:01.153685 29440 raft_consensus.cc:2258] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:01.153995 29439 raft_consensus.cc:2229] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:41:01.154417 29439 raft_consensus.cc:2258] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:01.154587 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 28796
I20260807 18:41:01.155300 29440 ts_tablet_manager.cc:1905] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:01.158363 29440 ts_tablet_manager.cc:1918] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:01.158473 29440 log.cc:1196] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/7e96ad527c6f428a86e8803d62e2cfbc
I20260807 18:41:01.158808 29440 ts_tablet_manager.cc:1939] T 7e96ad527c6f428a86e8803d62e2cfbc P b0fb355706ad47a7ba33269b6824378d: Deleting consensus metadata
I20260807 18:41:01.161540 28673 catalog_manager.cc:4841] TS b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457): tablet 7e96ad527c6f428a86e8803d62e2cfbc (table test-table [id=60ba9afc7ef44840a3dc44d39a419b1b]) successfully deleted
I20260807 18:41:01.162329 29439 ts_tablet_manager.cc:1905] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:01.165171 29439 ts_tablet_manager.cc:1918] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:01.165289 29439 log.cc:1196] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/7e96ad527c6f428a86e8803d62e2cfbc
I20260807 18:41:01.165680 29439 ts_tablet_manager.cc:1939] T 7e96ad527c6f428a86e8803d62e2cfbc P d5f36c29f6f2487ba435aac2aea35602: Deleting consensus metadata
I20260807 18:41:01.167032 28674 catalog_manager.cc:4841] TS d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513): tablet 7e96ad527c6f428a86e8803d62e2cfbc (table test-table [id=60ba9afc7ef44840a3dc44d39a419b1b]) successfully deleted
W20260807 18:41:01.170193 28674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:44771 (error 108)
I20260807 18:41:01.170944 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 28985
I20260807 18:41:01.188439 29395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.187879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:44771 (local address 127.0.0.1:59796)
0807 18:41:01.188172 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:01.188315 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:44771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
W20260807 18:41:01.188651 28674 catalog_manager.cc:4584] TS d6a3df0b65c64797b1d1d1d89cf996a2 (127.6.138.65:44771): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e96ad527c6f428a86e8803d62e2cfbc: Network error: Client connection negotiation failed: client connection to 127.6.138.65:44771: connect: Connection refused (error 111)
I20260807 18:41:01.194334 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 29193
I20260807 18:41:01.223752 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 28640
I20260807 18:41:01.239178 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37861
--webserver_interface=127.6.138.126
--webserver_port=40109
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:37861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:01.358040 29445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:01.358402 29445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:01.358477 29445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:01.362810 29445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:41:01.362920 29445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:01.362954 29445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:01.363003 29445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:41:01.363039 29445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:41:01.369186 29445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:37861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:01.370517 29445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:01.371881 29445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:01.392311 29461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:01.392300 29459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:01.392302 29458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:01.392823 29445 server_base.cc:1034] running on GCE node
I20260807 18:41:01.393549 29445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:01.394896 29445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:01.396080 29445 hybrid_clock.cc:648] HybridClock initialized: now 1786128061396050 us; error 46 us; skew 500 ppm
Aug 07 18:41:01 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128061, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:01.399326 29445 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:41:01.400381 29445 webserver.cc:460] Webserver started at http://127.6.138.126:40109/ using document root <none> and password file <none>
I20260807 18:41:01.400977 29445 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:01.401052 29445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:01.405439 29445 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:41:01.407917 29478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:01.409341 29445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:41:01.409452 29445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "9fcf9b79cf844a3a980bbd218a63dfcc"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "74d63a14f6e1c5806b4932180d5ad0f0"
server_key_iv: "5bd9b3c5caf8a47602738e677f0da31d"
server_key_version: "encryptionkey@0"
I20260807 18:41:01.410071 29445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:01.438364 29445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:01.442144 29445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:01.442328 29445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:01.450649 29445 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:37861
I20260807 18:41:01.450986 29555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:37861 every 8 connection(s)
I20260807 18:41:01.451934 29445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:41:01.456804 29558 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:41:01.457460 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 29445
I20260807 18:41:01.457979 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:44771
--local_ip_for_outbound_sockets=127.6.138.65
--tserver_master_addrs=127.6.138.126:37861
--webserver_port=36403
--webserver_interface=127.6.138.65
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:41:01.459264 29558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Bootstrap starting.
I20260807 18:41:01.477818 29558 log.cc:824] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:01.487558 29558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:41:01.488023 29558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Bootstrap complete.
I20260807 18:41:01.492539 29558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:01.492965 29558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fcf9b79cf844a3a980bbd218a63dfcc, State: Initialized, Role: FOLLOWER
I20260807 18:41:01.493570 29558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:01.493709 29558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:41:01.493803 29558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:41:01.493914 29558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:41:01.495986 29558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:01.496431 29558 leader_election.cc:302] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fcf9b79cf844a3a980bbd218a63dfcc; no voters:
I20260807 18:41:01.496811 29558 leader_election.cc:288] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:41:01.496945 29564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:41:01.497197 29564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 2 LEADER]: Becoming Leader. State: Replica: 9fcf9b79cf844a3a980bbd218a63dfcc, State: Running, Role: LEADER
I20260807 18:41:01.497558 29564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:01.498191 29558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:41:01.498638 29564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fcf9b79cf844a3a980bbd218a63dfcc. Latest consensus state: current_term: 2 leader_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } } }
I20260807 18:41:01.498749 29564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:01.499043 29565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } } }
I20260807 18:41:01.499166 29565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:01.499513 29575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:41:01.504817 29575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:41:01.505455 29575 catalog_manager.cc:1259] Loaded cluster ID: ea6b0d511d5549da942d24999b2ade8b
I20260807 18:41:01.505523 29575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:41:01.506991 29575 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 18:41:01.511828 29575 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 18:41:01.517935 29575 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:41:01.518800 29575 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Loaded TSK: 0
I20260807 18:41:01.520030 29575 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:41:01.581756 29560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:01.582078 29560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:01.582159 29560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:01.586288 29560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:01.586385 29560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:01.586479 29560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:41:01.592072 29560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:44771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=36403
--tserver_master_addrs=127.6.138.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:01.593190 29560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:01.594228 29560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:01.605154 29605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:01.605230 29604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:01.605154 29608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:01.605716 29560 server_base.cc:1034] running on GCE node
I20260807 18:41:01.606145 29560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:01.606807 29560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:01.607999 29560 hybrid_clock.cc:648] HybridClock initialized: now 1786128061607978 us; error 46 us; skew 500 ppm
Aug 07 18:41:01 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128061, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:01.610980 29560 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:41:01.612039 29560 webserver.cc:460] Webserver started at http://127.6.138.65:36403/ using document root <none> and password file <none>
I20260807 18:41:01.612694 29560 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:01.612773 29560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:01.617249 29560 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:41:01.619601 29621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:01.621362 29560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:01.621505 29560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "d6a3df0b65c64797b1d1d1d89cf996a2"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "b6a3d518014b4a2833da859e6d4830b7"
server_key_iv: "f2cf48bae1cff0a529a3d60bc0ff7474"
server_key_version: "encryptionkey@0"
I20260807 18:41:01.622256 29560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:01.650174 29560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:01.653987 29560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:01.654187 29560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:01.654887 29560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:01.656183 29634 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 18:41:01.660435 29560 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:41:01.660513 29560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260807 18:41:01.660668 29560 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:41:01.662732 29560 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:41:01.662813 29560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 18:41:01.663118 29634 tablet_bootstrap.cc:492] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Bootstrap starting.
I20260807 18:41:01.673940 29560 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:44771
I20260807 18:41:01.674031 29796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:44771 every 8 connection(s)
I20260807 18:41:01.674988 29560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:41:01.676868 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 29560
I20260807 18:41:01.677373 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:34457
--local_ip_for_outbound_sockets=127.6.138.66
--tserver_master_addrs=127.6.138.126:37861
--webserver_port=34633
--webserver_interface=127.6.138.66
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:41:01.684541 29634 log.cc:824] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:41:01 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128061, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:01.693338 29634 tablet_bootstrap.cc:492] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:41:01.693867 29634 tablet_bootstrap.cc:492] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Bootstrap complete.
I20260807 18:41:01.695441 29634 ts_tablet_manager.cc:1397] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.011s
I20260807 18:41:01.696408 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.677200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:44109)
0807 18:41:01.677951 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:01.678006 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:01.678926 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:01.680486 (+ 1560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:01.680499 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:01.681030 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:01.681816 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.681847 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.683403 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.683406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.684437 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.684454 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.685473 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.685479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.685741 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.686525 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:01.686555 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:01.688700 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:01.691713 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:01.691720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:01.691732 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:01.692150 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:01.692676 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:01.692679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:01.692681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:01.692860 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:01.694415 (+ 1555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:01.694429 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:01.694775 (+ 346us) client_negotiation.cc:769] Sending connection context
0807 18:41:01.694977 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 18:41:01.695367 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":165,"threads_started":1}
I20260807 18:41:01.696437 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.677203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:44109 (local address 127.6.138.126:37861)
0807 18:41:01.677638 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:01.677659 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:01.678143 (+ 484us) server_negotiation.cc:407] Connection header received
0807 18:41:01.679420 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:01.679435 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:01.679905 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:01.680351 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:01.682072 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.683202 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.684612 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.685299 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.685487 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.688964 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:01.688975 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:01.688997 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:01.689044 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:01.691548 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:01.692344 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:01.692351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:01.692358 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:01.692552 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:01.692994 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:01.693000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:01.693004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:01.693542 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:01.693683 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:01.695161 (+ 1478us) server_negotiation.cc:299] Negotiation successful
0807 18:41:01.695485 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":139,"threads_started":1}
I20260807 18:41:01.698498 29634 raft_consensus.cc:348] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.699074 29634 raft_consensus.cc:729] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a3df0b65c64797b1d1d1d89cf996a2, State: Initialized, Role: FOLLOWER
I20260807 18:41:01.699545 29797 heartbeater.cc:347] Connected to a master server at 127.6.138.126:37861
I20260807 18:41:01.699623 29634 consensus_queue.cc:260] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:01.699805 29797 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:01.700424 29797 heartbeater.cc:511] Master 127.6.138.126:37861 requested a full tablet report, sending...
I20260807 18:41:01.700755 29634 ts_tablet_manager.cc:1428] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Time spent starting tablet: real 0.005s user 0.006s sys 0.002s
I20260807 18:41:01.703174 29503 ts_manager.cc:194] Registered new tserver with Master: d6a3df0b65c64797b1d1d1d89cf996a2 (127.6.138.65:44771)
I20260807 18:41:01.707340 29503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:44109
I20260807 18:41:01.709918 29797 heartbeater.cc:503] Master 127.6.138.126:37861 was elected leader, sending a full tablet report...
Aug 07 18:41:01 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128061, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:41:01.718029 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.706754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:44771 (local address 127.0.0.1:59806)
0807 18:41:01.707488 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:01.707553 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:01.707726 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:01.708331 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:01.708337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:01.708802 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:01.708935 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.708944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.710293 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.710298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.710944 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.710953 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.711623 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.711627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.711740 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.712448 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:01.712461 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:01.713962 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:01.716072 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:01.716082 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:01.716088 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:01.716428 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:01.716754 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:01.716758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:01.716760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:01.716851 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:01.717325 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:01.717331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:01.717463 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 18:41:01.717619 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 18:41:01.717837 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":133,"threads_started":1}
I20260807 18:41:01.718065 29808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.706801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.6.138.65:44771)
0807 18:41:01.707221 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:01.707233 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:01.707658 (+ 425us) server_negotiation.cc:407] Connection header received
0807 18:41:01.707897 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:01.707902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:01.708153 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:01.708318 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:01.709085 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.710162 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.711081 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.711446 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.711551 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.714118 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:01.714132 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:01.714143 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:01.714165 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:01.715927 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:01.716553 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:01.716561 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:01.716566 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:01.716654 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:01.716952 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:01.716955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:01.716957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:01.717205 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:01.717350 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:01.717728 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 18:41:01.717896 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":107,"threads_started":1}
I20260807 18:41:01.719478 29694 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e96ad527c6f428a86e8803d62e2cfbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:01 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:59806
I20260807 18:41:01.720441 29811 tablet_replica.cc:330] stopping tablet replica
I20260807 18:41:01.720779 29811 raft_consensus.cc:2229] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:41:01.721051 29811 raft_consensus.cc:2258] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:01.722268 29811 ts_tablet_manager.cc:1905] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:01.727013 29811 ts_tablet_manager.cc:1918] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:01.727142 29811 log.cc:1196] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/7e96ad527c6f428a86e8803d62e2cfbc
I20260807 18:41:01.727492 29811 ts_tablet_manager.cc:1939] T 7e96ad527c6f428a86e8803d62e2cfbc P d6a3df0b65c64797b1d1d1d89cf996a2: Deleting consensus metadata
I20260807 18:41:01.729647 29484 catalog_manager.cc:4841] TS d6a3df0b65c64797b1d1d1d89cf996a2 (127.6.138.65:44771): tablet 7e96ad527c6f428a86e8803d62e2cfbc (table test-table [id=60ba9afc7ef44840a3dc44d39a419b1b]) successfully deleted
W20260807 18:41:01.810297 29801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:01.810609 29801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:01.810657 29801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:01.815035 29801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:01.815119 29801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:01.815182 29801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:41:01.820897 29801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:34457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=34633
--tserver_master_addrs=127.6.138.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:01.822084 29801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:01.823134 29801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:41:01.834228 29801 server_base.cc:1034] running on GCE node
W20260807 18:41:01.834184 29818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:01.834129 29816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:01.834161 29820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:01.835011 29801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:01.835712 29801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:01.836896 29801 hybrid_clock.cc:648] HybridClock initialized: now 1786128061836868 us; error 46 us; skew 500 ppm
Aug 07 18:41:01 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128061, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:01.840027 29801 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:41:01.841164 29801 webserver.cc:460] Webserver started at http://127.6.138.66:34633/ using document root <none> and password file <none>
I20260807 18:41:01.841799 29801 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:01.841921 29801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:01.846261 29801 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 18:41:01.848876 29828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:01.850406 29801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:01.850581 29801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "b0fb355706ad47a7ba33269b6824378d"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "1e6a5d4228d7a1764c435388cb5c9ce7"
server_key_iv: "30d3804db41e586c406a6fe692d77cb0"
server_key_version: "encryptionkey@0"
I20260807 18:41:01.851236 29801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:01.870400 29801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:01.874483 29801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:01.874841 29801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:01.875507 29801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:01.876546 29801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:01.876602 29801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:01.876645 29801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:01.876684 29801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:01.886513 29801 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:34457
I20260807 18:41:01.886611 29989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:34457 every 8 connection(s)
I20260807 18:41:01.887634 29801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:41:01.888211 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 29801
I20260807 18:41:01.888747 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:44513
--local_ip_for_outbound_sockets=127.6.138.67
--tserver_master_addrs=127.6.138.126:37861
--webserver_port=33789
--webserver_interface=127.6.138.67
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:41:01 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128061, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:01.904409 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.890070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:54949 (local address 127.6.138.126:37861)
0807 18:41:01.890404 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:01.890409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:01.891481 (+ 1072us) server_negotiation.cc:407] Connection header received
0807 18:41:01.892976 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:01.892980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:01.893057 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:01.893160 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:01.894451 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.894987 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.896199 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.896512 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.896633 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.899593 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:01.899599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:01.899603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:01.899645 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:01.901761 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:01.902563 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:01.902568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:01.902570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:01.902678 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:01.903085 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:01.903089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:01.903091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:01.903342 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:01.903445 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:01.904029 (+ 584us) server_negotiation.cc:299] Negotiation successful
0807 18:41:01.904191 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:41:01.905375 29995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:01.890616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:54949)
0807 18:41:01.891211 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:01.891259 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:01.892722 (+ 1463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:01.893374 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:01.893382 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:01.893775 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:01.894288 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.894303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.895141 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.895144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.896052 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:01.896062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:01.896692 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:01.896696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:01.896873 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:01.897600 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:01.897618 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:01.899345 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:01.901951 (+ 2606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:01.901961 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:01.901984 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:01.902440 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:01.902806 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:01.902809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:01.902810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:01.902982 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:01.903451 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:01.903457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:01.903752 (+ 295us) client_negotiation.cc:769] Sending connection context
0807 18:41:01.903928 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 18:41:01.904396 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":165,"threads_started":1}
I20260807 18:41:01.906852 29990 heartbeater.cc:347] Connected to a master server at 127.6.138.126:37861
I20260807 18:41:01.907140 29990 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:01.907750 29990 heartbeater.cc:511] Master 127.6.138.126:37861 requested a full tablet report, sending...
I20260807 18:41:01.908982 29503 ts_manager.cc:194] Registered new tserver with Master: b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457)
I20260807 18:41:01.909792 29503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:54949
W20260807 18:41:02.016201 29993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:02.016566 29993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:02.016613 29993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:02.021009 29993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:02.021085 29993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:02.021148 29993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:41:02.026733 29993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:44513
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=33789
--tserver_master_addrs=127.6.138.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:02.028002 29993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:02.029182 29993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:02.039572 30001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:02.039590 30002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:02.039590 30004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:02.039705 29993 server_base.cc:1034] running on GCE node
I20260807 18:41:02.040383 29993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:02.041088 29993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:02.042302 29993 hybrid_clock.cc:648] HybridClock initialized: now 1786128062042255 us; error 59 us; skew 500 ppm
Aug 07 18:41:02 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128062, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:02.045594 29993 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:41:02.046672 29993 webserver.cc:460] Webserver started at http://127.6.138.67:33789/ using document root <none> and password file <none>
I20260807 18:41:02.047324 29993 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:02.047461 29993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:02.051916 29993 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:02.054302 30011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:02.055900 29993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:41:02.056066 29993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "d5f36c29f6f2487ba435aac2aea35602"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "98a9789362fed2da9e0defdb5bb70bd5"
server_key_iv: "885370df6d8c454de0692be191b2d81b"
server_key_version: "encryptionkey@0"
I20260807 18:41:02.056761 29993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:02.073536 29993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:02.077325 29993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:02.077697 29993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:02.078369 29993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:02.079284 29993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:02.079356 29993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:02.079434 29993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:02.079478 29993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:02.089438 29993 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:44513
I20260807 18:41:02.089497 30177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:44513 every 8 connection(s)
I20260807 18:41:02.090620 29993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:41:02.097750 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 29993
Aug 07 18:41:02 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128062, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:02.106721 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.092894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:39315 (local address 127.6.138.126:37861)
0807 18:41:02.093066 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:02.093093 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:02.093978 (+ 885us) server_negotiation.cc:407] Connection header received
0807 18:41:02.095336 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:02.095339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:02.095394 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:02.095520 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:02.096868 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.097447 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.099154 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.099485 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.099601 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.102292 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:02.102296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:02.102299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:02.102362 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:02.104189 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:02.104891 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:02.104894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:02.104895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:02.104976 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:02.105433 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:02.105437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:02.105439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:02.105628 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:02.105739 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:02.106420 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 18:41:02.106548 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:41:02.107609 30184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.098548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.6.138.126:37861)
0807 18:41:02.098844 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:02.098848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:02.099029 (+ 181us) server_negotiation.cc:407] Connection header received
0807 18:41:02.099194 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:02.099197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:02.099240 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:02.099308 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:02.100361 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.100869 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.101784 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.102134 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.102255 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.104109 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:02.104112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:02.104114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:02.104137 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:02.105831 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:02.106319 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:02.106322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:02.106323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:02.106402 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:02.106729 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:02.106736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:02.106738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:02.106964 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:02.107192 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:02.107357 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 18:41:02.107457 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"spinlock_wait_cycles":138624,"thread_start_us":118,"threads_started":1}
I20260807 18:41:02.107923 30182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.093286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:39315)
0807 18:41:02.093793 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.093841 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:02.095062 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:02.095694 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:02.095701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:02.096120 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:02.096678 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.096693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.097640 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.097645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.099015 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.099028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.099647 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.099650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.099855 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.100542 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:02.100558 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:02.102119 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:02.104331 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:02.104340 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:02.104358 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:02.104781 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:02.105156 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:02.105160 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:02.105162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:02.105322 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:02.105739 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:02.105748 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:02.106097 (+ 349us) client_negotiation.cc:769] Sending connection context
0807 18:41:02.106356 (+ 259us) client_negotiation.cc:240] Negotiation successful
0807 18:41:02.106723 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260807 18:41:02.109148 30180 heartbeater.cc:347] Connected to a master server at 127.6.138.126:37861
I20260807 18:41:02.109416 30180 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:02.109951 30180 heartbeater.cc:511] Master 127.6.138.126:37861 requested a full tablet report, sending...
I20260807 18:41:02.111035 29503 ts_manager.cc:194] Registered new tserver with Master: d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513)
I20260807 18:41:02.111681 29503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:39315
I20260807 18:41:02.121744 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:41:02.126499 30184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.122532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.6.138.126:37861)
0807 18:41:02.122674 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:02.122678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:02.122782 (+ 104us) server_negotiation.cc:407] Connection header received
0807 18:41:02.122926 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:02.122928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:02.123036 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:02.123133 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:41:02.123406 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.123979 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.124981 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.125250 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.125346 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.125646 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:41:02.126084 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:41:02.126187 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:02.126317 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 18:41:02.126379 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:41:02.127255 29503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:41:02.131210 29503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:41:02.140671 29694 tablet_service.cc:1467] Processing CreateTablet for tablet e3bfa3997a3a428a8bd371c0af2b9538 (DEFAULT_TABLE table=test-table [id=d9c6fd4504354f7d864c86e6b4424c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:02.141461 29694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3bfa3997a3a428a8bd371c0af2b9538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:02.144619 29634 tablet_bootstrap.cc:492] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Bootstrap starting.
I20260807 18:41:02.144994 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.138868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:34457 (local address 127.0.0.1:57452)
0807 18:41:02.138994 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.139012 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:02.139166 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:02.140203 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:02.140206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:02.140225 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:02.140375 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.140385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.141973 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.141977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.142962 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.142973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.144492 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.144496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.144615 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.144682 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 18:41:02.144767 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:41:02.144859 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 18:41:02.145342 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.139479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54832 (local address 127.6.138.67:44513)
0807 18:41:02.139924 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:02.139937 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:02.139969 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:41:02.140034 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:02.140038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:02.140251 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:02.140430 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:02.141315 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.142303 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.143722 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.144746 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.144848 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.144989 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:02.145088 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 18:41:02.145197 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":117,"threads_started":1}
I20260807 18:41:02.145435 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.139099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57452 (local address 127.6.138.66:34457)
0807 18:41:02.139566 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:02.139581 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:02.139613 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:41:02.139708 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:02.139717 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:02.140019 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:02.140213 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:02.140665 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.141815 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.143103 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.144347 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.144454 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.144616 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:02.145190 (+ 574us) server_negotiation.cc:299] Negotiation successful
0807 18:41:02.145295 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":238,"threads_started":1}
I20260807 18:41:02.145608 29634 tablet_bootstrap.cc:654] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:02.145746 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.139231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:44513 (local address 127.0.0.1:54832)
0807 18:41:02.139690 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.139705 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:02.139799 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:02.140500 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:02.140502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:02.140519 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:02.140608 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.140755 (+ 147us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.142477 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.142481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.143577 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.143589 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.144863 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.144867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.144957 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.144979 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:41:02.145278 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 18:41:02.145367 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":104,"threads_started":1}
I20260807 18:41:02.147224 29634 tablet_bootstrap.cc:492] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: No bootstrap required, opened a new log
I20260807 18:41:02.147337 29634 ts_tablet_manager.cc:1397] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:41:02.147753 29634 raft_consensus.cc:348] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.147876 29634 raft_consensus.cc:374] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:02.147921 29634 raft_consensus.cc:729] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a3df0b65c64797b1d1d1d89cf996a2, State: Initialized, Role: FOLLOWER
I20260807 18:41:02.148084 29634 consensus_queue.cc:260] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.148377 29634 ts_tablet_manager.cc:1428] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:41:02.149189 30082 tablet_service.cc:1467] Processing CreateTablet for tablet e3bfa3997a3a428a8bd371c0af2b9538 (DEFAULT_TABLE table=test-table [id=d9c6fd4504354f7d864c86e6b4424c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:02.149188 29900 tablet_service.cc:1467] Processing CreateTablet for tablet e3bfa3997a3a428a8bd371c0af2b9538 (DEFAULT_TABLE table=test-table [id=d9c6fd4504354f7d864c86e6b4424c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:02.150411 30082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3bfa3997a3a428a8bd371c0af2b9538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:02.150409 29900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3bfa3997a3a428a8bd371c0af2b9538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:02.159394 30196 tablet_bootstrap.cc:492] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: Bootstrap starting.
I20260807 18:41:02.159934 30197 tablet_bootstrap.cc:492] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: Bootstrap starting.
I20260807 18:41:02.162349 30196 tablet_bootstrap.cc:654] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:02.162819 30197 tablet_bootstrap.cc:654] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:02.163751 30197 log.cc:824] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:02.163751 30196 log.cc:824] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:02.166165 30197 tablet_bootstrap.cc:492] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: No bootstrap required, opened a new log
I20260807 18:41:02.166214 30196 tablet_bootstrap.cc:492] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: No bootstrap required, opened a new log
I20260807 18:41:02.166391 30196 ts_tablet_manager.cc:1397] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 18:41:02.166392 30197 ts_tablet_manager.cc:1397] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 18:41:02.170617 30196 raft_consensus.cc:348] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.170617 30197 raft_consensus.cc:348] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.170961 30197 raft_consensus.cc:374] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:02.170961 30196 raft_consensus.cc:374] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:02.171016 30197 raft_consensus.cc:729] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fb355706ad47a7ba33269b6824378d, State: Initialized, Role: FOLLOWER
I20260807 18:41:02.171016 30196 raft_consensus.cc:729] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f36c29f6f2487ba435aac2aea35602, State: Initialized, Role: FOLLOWER
I20260807 18:41:02.171536 30197 consensus_queue.cc:260] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.171536 30196 consensus_queue.cc:260] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.172436 30180 heartbeater.cc:503] Master 127.6.138.126:37861 was elected leader, sending a full tablet report...
I20260807 18:41:02.172767 30196 ts_tablet_manager.cc:1428] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260807 18:41:02.172801 29990 heartbeater.cc:503] Master 127.6.138.126:37861 was elected leader, sending a full tablet report...
I20260807 18:41:02.173676 30197 ts_tablet_manager.cc:1428] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260807 18:41:02.176350 29798 tablet.cc:2369] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:41:02.227947 29805 raft_consensus.cc:482] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:41:02.228225 29805 raft_consensus.cc:504] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.229660 29805 leader_election.cc:288] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457), d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513)
I20260807 18:41:02.233875 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.230167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:51615 (local address 127.6.138.67:44513)
0807 18:41:02.230302 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:02.230308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:02.230420 (+ 112us) server_negotiation.cc:407] Connection header received
0807 18:41:02.230630 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:02.230637 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:02.230698 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:02.230760 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:02.231071 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.231619 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.232764 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.233184 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.233282 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.233323 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:02.233656 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 18:41:02.233738 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:41:02.233887 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.230320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:34915 (local address 127.6.138.66:34457)
0807 18:41:02.230445 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:02.230450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:02.230526 (+ 76us) server_negotiation.cc:407] Connection header received
0807 18:41:02.230700 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:02.230702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:02.230773 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:02.230872 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:02.231169 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.231656 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.232829 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.233279 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.233400 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.233459 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:02.233703 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 18:41:02.233780 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 18:41:02.233880 30204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.230019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:34457 (local address 127.6.138.65:34915)
0807 18:41:02.230425 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.230442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:02.230573 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:02.230917 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:02.230919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:02.230935 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:02.231045 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.231052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.231807 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.231810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.232664 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.232673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.233414 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.233417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.233552 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.233569 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 18:41:02.233665 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:41:02.233737 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":48,"thread_start_us":93,"threads_started":1}
I20260807 18:41:02.233887 30205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.229946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:44513 (local address 127.6.138.65:51615)
0807 18:41:02.230301 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.230318 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:02.230462 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:02.230800 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:02.230804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:02.230825 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:02.230947 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.230954 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.231739 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.231744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.232633 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:02.232644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.233315 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.233322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:02.233416 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.233544 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 18:41:02.233608 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 18:41:02.233717 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":24,"thread_start_us":78,"threads_started":1}
I20260807 18:41:02.234723 30114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3bfa3997a3a428a8bd371c0af2b9538" candidate_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f36c29f6f2487ba435aac2aea35602" is_pre_election: true
I20260807 18:41:02.234723 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3bfa3997a3a428a8bd371c0af2b9538" candidate_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fb355706ad47a7ba33269b6824378d" is_pre_election: true
I20260807 18:41:02.235092 29930 raft_consensus.cc:2454] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a3df0b65c64797b1d1d1d89cf996a2 in term 0.
I20260807 18:41:02.235092 30114 raft_consensus.cc:2454] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6a3df0b65c64797b1d1d1d89cf996a2 in term 0.
I20260807 18:41:02.235718 29631 leader_election.cc:302] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b0fb355706ad47a7ba33269b6824378d, d5f36c29f6f2487ba435aac2aea35602, d6a3df0b65c64797b1d1d1d89cf996a2; no voters:
I20260807 18:41:02.236126 29805 raft_consensus.cc:2781] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:41:02.236279 29805 raft_consensus.cc:482] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:41:02.236377 29805 raft_consensus.cc:3037] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:02.237563 29805 raft_consensus.cc:504] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.237974 29805 leader_election.cc:288] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 1 election: Requested vote from peers b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457), d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513)
I20260807 18:41:02.238447 30114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3bfa3997a3a428a8bd371c0af2b9538" candidate_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f36c29f6f2487ba435aac2aea35602"
I20260807 18:41:02.238602 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3bfa3997a3a428a8bd371c0af2b9538" candidate_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0fb355706ad47a7ba33269b6824378d"
I20260807 18:41:02.238646 30114 raft_consensus.cc:3037] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:02.238759 29930 raft_consensus.cc:3037] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:02.240221 30114 raft_consensus.cc:2454] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a3df0b65c64797b1d1d1d89cf996a2 in term 1.
I20260807 18:41:02.240235 29930 raft_consensus.cc:2454] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a3df0b65c64797b1d1d1d89cf996a2 in term 1.
I20260807 18:41:02.240654 29631 leader_election.cc:302] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5f36c29f6f2487ba435aac2aea35602, d6a3df0b65c64797b1d1d1d89cf996a2; no voters:
I20260807 18:41:02.240914 29805 raft_consensus.cc:2781] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:41:02.241163 29805 raft_consensus.cc:686] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 LEADER]: Becoming Leader. State: Replica: d6a3df0b65c64797b1d1d1d89cf996a2, State: Running, Role: LEADER
I20260807 18:41:02.241472 29805 consensus_queue.cc:237] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.244800 29504 catalog_manager.cc:5495] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6a3df0b65c64797b1d1d1d89cf996a2 (127.6.138.65). New cstate: current_term: 1 leader_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } health_report { overall_health: HEALTHY } } }
I20260807 18:41:02.277890 30211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.273734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.6.138.65:44771)
0807 18:41:02.274106 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:02.274114 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:02.274128 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:41:02.274208 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:02.274210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:02.274265 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:02.274391 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:41:02.274711 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.275283 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.276207 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:02.276531 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:02.276630 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:02.276936 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:41:02.277385 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:41:02.277476 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:02.277616 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 18:41:02.277697 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":124,"threads_started":1}
I20260807 18:41:02.282569 29930 raft_consensus.cc:1261] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [term 1 FOLLOWER]: Refusing update from remote peer d6a3df0b65c64797b1d1d1d89cf996a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:41:02.282588 30114 raft_consensus.cc:1261] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [term 1 FOLLOWER]: Refusing update from remote peer d6a3df0b65c64797b1d1d1d89cf996a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:41:02.283439 29805 consensus_queue.cc:1035] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:41:02.283664 30207 consensus_queue.cc:1035] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:41:02.293147 30212 mvcc.cc:205] Tried to move back new op lower bound from 7315980543101644800 to 7315980542944227328. Current Snapshot: MvccSnapshot[applied={T|T < 7315980543101644800}]
I20260807 18:41:02.295110 30214 mvcc.cc:205] Tried to move back new op lower bound from 7315980543101644800 to 7315980542944227328. Current Snapshot: MvccSnapshot[applied={T|T < 7315980543101644800}]
I20260807 18:41:02.295477 30213 mvcc.cc:205] Tried to move back new op lower bound from 7315980543101644800 to 7315980542944227328. Current Snapshot: MvccSnapshot[applied={T|T < 7315980543101644800}]
I20260807 18:41:02.302965 29504 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:41:02.303179 29504 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:41:02.305892 29504 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Sending DeleteTablet for 3 replicas of tablet e3bfa3997a3a428a8bd371c0af2b9538
I20260807 18:41:02.306691 29900 tablet_service.cc:1514] Processing DeleteTablet for tablet e3bfa3997a3a428a8bd371c0af2b9538 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:02 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:57452
I20260807 18:41:02.306872 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 29560
I20260807 18:41:02.306838 30082 tablet_service.cc:1514] Processing DeleteTablet for tablet e3bfa3997a3a428a8bd371c0af2b9538 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:02 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:54832
I20260807 18:41:02.314829 30231 tablet_replica.cc:330] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: stopping tablet replica
I20260807 18:41:02.315121 30228 tablet_replica.cc:330] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: stopping tablet replica
I20260807 18:41:02.315356 30231 raft_consensus.cc:2229] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:41:02.315470 30228 raft_consensus.cc:2229] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:41:02.315765 30231 raft_consensus.cc:2258] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:02.315788 30228 raft_consensus.cc:2258] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:02.317297 30231 ts_tablet_manager.cc:1905] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:02.317662 30228 ts_tablet_manager.cc:1905] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:02.319767 30231 ts_tablet_manager.cc:1918] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:02.319902 30231 log.cc:1196] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/e3bfa3997a3a428a8bd371c0af2b9538
I20260807 18:41:02.320327 30231 ts_tablet_manager.cc:1939] T e3bfa3997a3a428a8bd371c0af2b9538 P d5f36c29f6f2487ba435aac2aea35602: Deleting consensus metadata
I20260807 18:41:02.321233 30228 ts_tablet_manager.cc:1918] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:02.321354 30228 log.cc:1196] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/e3bfa3997a3a428a8bd371c0af2b9538
I20260807 18:41:02.321628 29484 catalog_manager.cc:4841] TS d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513): tablet e3bfa3997a3a428a8bd371c0af2b9538 (table test-table [id=d9c6fd4504354f7d864c86e6b4424c50]) successfully deleted
I20260807 18:41:02.321702 30228 ts_tablet_manager.cc:1939] T e3bfa3997a3a428a8bd371c0af2b9538 P b0fb355706ad47a7ba33269b6824378d: Deleting consensus metadata
I20260807 18:41:02.323091 29481 catalog_manager.cc:4841] TS b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457): tablet e3bfa3997a3a428a8bd371c0af2b9538 (table test-table [id=d9c6fd4504354f7d864c86e6b4424c50]) successfully deleted
W20260807 18:41:02.323693 29484 connection.cc:535] client connection to 127.6.138.65:44771 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:41:02.323800 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 29801
W20260807 18:41:02.323860 29484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:41:02.328236 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.327766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:44771 (local address 127.0.0.1:59834)
0807 18:41:02.328016 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.328146 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:44771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260807 18:41:02.328615 29484 catalog_manager.cc:4584] TS d6a3df0b65c64797b1d1d1d89cf996a2 (127.6.138.65:44771): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3bfa3997a3a428a8bd371c0af2b9538: Network error: Client connection negotiation failed: client connection to 127.6.138.65:44771: connect: Connection refused (error 111)
I20260807 18:41:02.334276 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 29993
I20260807 18:41:02.354521 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 29445
I20260807 18:41:02.367570 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37861
--webserver_interface=127.6.138.126
--webserver_port=40109
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:37861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:02.485316 30234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:02.485651 30234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:02.485728 30234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:02.489846 30234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:41:02.489967 30234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:02.490003 30234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:02.490046 30234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:41:02.490090 30234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:41:02.495946 30234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:37861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:02.497278 30234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:02.498452 30234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:02.507613 30245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:02.507654 30247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:02.507645 30234 server_base.cc:1034] running on GCE node
W20260807 18:41:02.507817 30243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:02.508505 30234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:02.509687 30234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:02.510877 30234 hybrid_clock.cc:648] HybridClock initialized: now 1786128062510840 us; error 44 us; skew 500 ppm
Aug 07 18:41:02 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128062, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:02.513794 30234 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:41:02.514734 30234 webserver.cc:460] Webserver started at http://127.6.138.126:40109/ using document root <none> and password file <none>
I20260807 18:41:02.515342 30234 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:02.515440 30234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:02.519572 30234 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:41:02.521867 30261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:02.523144 30234 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:41:02.523295 30234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "9fcf9b79cf844a3a980bbd218a63dfcc"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "74d63a14f6e1c5806b4932180d5ad0f0"
server_key_iv: "5bd9b3c5caf8a47602738e677f0da31d"
server_key_version: "encryptionkey@0"
I20260807 18:41:02.523900 30234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:02.539230 30234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:02.542697 30234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:02.542932 30234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:02.550625 30234 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:37861
I20260807 18:41:02.550937 30328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:37861 every 8 connection(s)
I20260807 18:41:02.551819 30234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:41:02.555845 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 30234
I20260807 18:41:02.556387 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:44771
--local_ip_for_outbound_sockets=127.6.138.65
--tserver_master_addrs=127.6.138.126:37861
--webserver_port=36403
--webserver_interface=127.6.138.65
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:41:02.556707 30329 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:41:02.559168 30329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Bootstrap starting.
I20260807 18:41:02.577527 30329 log.cc:824] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:02.589066 30329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:41:02.589504 30329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc: Bootstrap complete.
I20260807 18:41:02.593784 30329 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:02.594177 30329 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fcf9b79cf844a3a980bbd218a63dfcc, State: Initialized, Role: FOLLOWER
I20260807 18:41:02.594666 30329 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:02.594781 30329 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:41:02.594866 30329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:41:02.594986 30329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:41:02.597020 30329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:02.597371 30329 leader_election.cc:302] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fcf9b79cf844a3a980bbd218a63dfcc; no voters:
I20260807 18:41:02.597723 30329 leader_election.cc:288] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:41:02.597844 30334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:41:02.598037 30334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [term 3 LEADER]: Becoming Leader. State: Replica: 9fcf9b79cf844a3a980bbd218a63dfcc, State: Running, Role: LEADER
I20260807 18:41:02.598338 30334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } }
I20260807 18:41:02.599015 30329 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:41:02.599359 30334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fcf9b79cf844a3a980bbd218a63dfcc. Latest consensus state: current_term: 3 leader_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } } }
I20260807 18:41:02.599483 30334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:02.599880 30339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:41:02.599905 30336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcf9b79cf844a3a980bbd218a63dfcc" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37861 } } }
I20260807 18:41:02.600138 30336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcf9b79cf844a3a980bbd218a63dfcc [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:02.605451 30339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:41:02.606274 30339 catalog_manager.cc:1259] Loaded cluster ID: ea6b0d511d5549da942d24999b2ade8b
I20260807 18:41:02.606355 30339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 18:41:02.607266 30339 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1faf43b9ad google::LogMessage::Fail() at ??:0
@ 0x7f1faf43e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1faf43b4de google::LogMessage::Flush() at ??:0
@ 0x7f1faf43ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1fb9d7afdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f1fb9d7bd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1fb9d78579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f1fb9dbc640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1fb168d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1fb168dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f1fb168f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f1fb167e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1fb2cc26db start_thread at ??:0
@ 0x7f1fadf3961f clone at ??:0
W20260807 18:41:02.706673 30331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:02.706943 30331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:02.706991 30331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:02.711093 30331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:02.711158 30331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:02.711217 30331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:41:02.716728 30331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:44771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=36403
--tserver_master_addrs=127.6.138.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:02.717864 30331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:02.718907 30331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:02.728973 30386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:02.728973 30387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:02.728979 30389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:02.729485 30331 server_base.cc:1034] running on GCE node
I20260807 18:41:02.729909 30331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:02.730571 30331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:02.731771 30331 hybrid_clock.cc:648] HybridClock initialized: now 1786128062731742 us; error 48 us; skew 500 ppm
Aug 07 18:41:02 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128062, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:02.734858 30331 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:41:02.735862 30331 webserver.cc:460] Webserver started at http://127.6.138.65:36403/ using document root <none> and password file <none>
I20260807 18:41:02.736537 30331 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:02.736637 30331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:02.740787 30331 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:02.743194 30397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:02.744689 30331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:02.744831 30331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "d6a3df0b65c64797b1d1d1d89cf996a2"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "b6a3d518014b4a2833da859e6d4830b7"
server_key_iv: "f2cf48bae1cff0a529a3d60bc0ff7474"
server_key_version: "encryptionkey@0"
I20260807 18:41:02.745412 30331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:02.778275 30331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:02.781749 30331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:02.781978 30331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:02.782649 30331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:02.783895 30406 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 18:41:02.789765 30331 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:41:02.789862 30331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 18:41:02.789973 30331 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:41:02.791741 30331 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:41:02.791821 30331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 18:41:02.792147 30406 tablet_bootstrap.cc:492] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Bootstrap starting.
I20260807 18:41:02.805480 30331 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:44771
I20260807 18:41:02.805929 30570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:44771 every 8 connection(s)
I20260807 18:41:02.806895 30331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
W20260807 18:41:02.813148 30574 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:41:02.809525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:51575)
0807 18:41:02.810138 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.810210 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:02.811583 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:02.811996 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":155,"threads_started":1}
W20260807 18:41:02.813640 30572 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:37861 (0 consecutive failures): Network error: Failed to ping master at 127.6.138.126:37861: Client connection negotiation failed: client connection to 127.6.138.126:37861: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260807 18:41:02.814363 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.814031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:59811)
0807 18:41:02.814198 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.814285 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:41:02.815054 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.814851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:55597)
0807 18:41:02.814958 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.815001 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 18:41:02.815299 30572 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:41:02.816135 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 30331
I20260807 18:41:02.816553 30406 log.cc:824] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:02.816592 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:34457
--local_ip_for_outbound_sockets=127.6.138.66
--tserver_master_addrs=127.6.138.126:37861
--webserver_port=34633
--webserver_interface=127.6.138.66
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:41:02.823895 30406 tablet_bootstrap.cc:492] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:41:02.824365 30406 tablet_bootstrap.cc:492] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Bootstrap complete.
I20260807 18:41:02.825644 30406 ts_tablet_manager.cc:1397] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Time spent bootstrapping tablet: real 0.034s user 0.014s sys 0.015s
I20260807 18:41:02.828145 30406 raft_consensus.cc:348] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.828730 30406 raft_consensus.cc:729] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a3df0b65c64797b1d1d1d89cf996a2, State: Initialized, Role: FOLLOWER
I20260807 18:41:02.829262 30406 consensus_queue.cc:260] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:02.830410 30406 ts_tablet_manager.cc:1428] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:41:02.830679 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.830366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:52593)
0807 18:41:02.830515 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.830591 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 18:41:02.831382 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:02.831156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:53233)
0807 18:41:02.831267 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:02.831320 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:41:02.937014 30577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:02.937292 30577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:02.937337 30577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:02.941438 30577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:02.941506 30577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:02.941602 30577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:41:02.947086 30577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:34457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=34633
--tserver_master_addrs=127.6.138.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:02.948203 30577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:02.949277 30577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:41:02.959301 30577 server_base.cc:1034] running on GCE node
W20260807 18:41:02.959227 30584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:02.959227 30586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:02.959254 30591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:02.959931 30577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:02.960647 30577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:02.961817 30577 hybrid_clock.cc:648] HybridClock initialized: now 1786128062961796 us; error 32 us; skew 500 ppm
Aug 07 18:41:02 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128062, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:02.964718 30577 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:41:02.965583 30577 webserver.cc:460] Webserver started at http://127.6.138.66:34633/ using document root <none> and password file <none>
I20260807 18:41:02.966159 30577 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:02.966224 30577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:02.970230 30577 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:41:02.972501 30601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:02.973824 30577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:41:02.973928 30577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "b0fb355706ad47a7ba33269b6824378d"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "1e6a5d4228d7a1764c435388cb5c9ce7"
server_key_iv: "30d3804db41e586c406a6fe692d77cb0"
server_key_version: "encryptionkey@0"
I20260807 18:41:02.974509 30577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:02.990199 30577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:02.993801 30577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:02.994035 30577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:02.994668 30577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:02.995536 30577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:02.995609 30577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:02.995683 30577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:02.995728 30577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:03.004380 30577 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:34457
I20260807 18:41:03.004432 30767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:34457 every 8 connection(s)
I20260807 18:41:03.005438 30577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:41:03.009418 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:03.007702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:34443)
0807 18:41:03.008191 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:03.008696 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":125,"threads_started":1}
W20260807 18:41:03.010012 30769 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:37861 (0 consecutive failures): Network error: Failed to ping master at 127.6.138.126:37861: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
I20260807 18:41:03.010690 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:03.010450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:44463)
0807 18:41:03.010569 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:03.010623 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 18:41:03.011305 30772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:03.011120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:55171)
0807 18:41:03.011218 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:03.011257 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 18:41:03.011493 30769 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:41:03.015443 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 30577
I20260807 18:41:03.015851 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:44513
--local_ip_for_outbound_sockets=127.6.138.67
--tserver_master_addrs=127.6.138.126:37861
--webserver_port=33789
--webserver_interface=127.6.138.67
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:03.135957 30774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:03.136320 30774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:03.136421 30774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:03.140609 30774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:03.140679 30774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:03.140784 30774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:41:03.146453 30774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:44513
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=33789
--tserver_master_addrs=127.6.138.126:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:03.147650 30774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:03.148767 30774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:03.159125 30783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:03.159158 30781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:03.159382 30774 server_base.cc:1034] running on GCE node
W20260807 18:41:03.159119 30787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:03.159967 30774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:03.160688 30774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:03.161883 30774 hybrid_clock.cc:648] HybridClock initialized: now 1786128063161844 us; error 54 us; skew 500 ppm
Aug 07 18:41:03 dist-test-slave-2d19 krb5kdc[28620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128063, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:03.164891 30774 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:41:03.165861 30774 webserver.cc:460] Webserver started at http://127.6.138.67:33789/ using document root <none> and password file <none>
I20260807 18:41:03.166466 30774 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:03.166571 30774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:03.170826 30774 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:41:03.173430 30796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:03.174958 30774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:03.175119 30774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "d5f36c29f6f2487ba435aac2aea35602"
format_stamp: "Formatted at 2026-08-07 18:41:00 on dist-test-slave-2d19"
server_key: "98a9789362fed2da9e0defdb5bb70bd5"
server_key_iv: "885370df6d8c454de0692be191b2d81b"
server_key_version: "encryptionkey@0"
I20260807 18:41:03.175738 30774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:03.204807 30774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:03.208410 30774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:03.208648 30774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:03.209313 30774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:03.210213 30774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:03.210287 30774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:03.210359 30774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:03.210412 30774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:03.219482 30774 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:44513
I20260807 18:41:03.219548 30952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:44513 every 8 connection(s)
I20260807 18:41:03.220647 30774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:41:03.224041 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 30774
I20260807 18:41:03.225333 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:03.223087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:58577)
0807 18:41:03.223641 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:03.224443 (+ 802us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":131,"threads_started":1}
W20260807 18:41:03.225813 30954 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:37861 (0 consecutive failures): Network error: Failed to ping master at 127.6.138.126:37861: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
I20260807 18:41:03.226440 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:03.226208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:41935)
0807 18:41:03.226323 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:03.226382 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 18:41:03.227152 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:03.226889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:40723)
0807 18:41:03.227052 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:03.227103 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260807 18:41:03.227367 30954 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:41:03.832996 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:03.832431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:53489)
0807 18:41:03.832791 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:03.832865 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":85,"threads_started":1}
I20260807 18:41:04.012642 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:04.012140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:41483)
0807 18:41:04.012470 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:04.012534 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":137,"threads_started":1}
I20260807 18:41:04.228382 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:04.227916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:35687)
0807 18:41:04.228164 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:04.228221 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":74,"threads_started":1}
I20260807 18:41:04.259869 30965 raft_consensus.cc:482] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:41:04.260113 30965 raft_consensus.cc:504] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:04.261513 30965 leader_election.cc:288] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457), d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513)
Aug 07 18:41:04 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128062, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.66@KRBTEST.COM
Aug 07 18:41:04 dist-test-slave-2d19 krb5kdc[28620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128062, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.67@KRBTEST.COM
I20260807 18:41:04.284087 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:04.261781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:34457 (local address 127.6.138.65:52531)
0807 18:41:04.261910 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:04.261926 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:04.262050 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:04.263843 (+ 1793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:04.263851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:04.264456 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:04.264940 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:04.264956 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:04.266286 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:04.266289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:04.267271 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:04.267287 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:04.268288 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:04.268292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:04.268496 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:04.275805 (+ 7309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:04.275837 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:04.277935 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:04.281125 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:04.281131 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:04.281144 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:04.281409 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:04.281792 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:04.281794 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:04.281795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:04.281941 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:04.282852 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:04.282860 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:04.283246 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 18:41:04.283484 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 18:41:04.283860 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":116,"spinlock_wait_cycles":6272}
I20260807 18:41:04.284130 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:04.261798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:44513 (local address 127.6.138.65:42999)
0807 18:41:04.262302 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:04.262315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:04.262417 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:04.263842 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:04.263850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:04.264466 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:04.264943 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:04.264956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:04.266273 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:04.266282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:04.267252 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:04.267266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:04.268580 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:04.268584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:04.268670 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:04.275808 (+ 7138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:04.275829 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:04.278029 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:04.281509 (+ 3480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:04.281511 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:04.281512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:04.281670 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:04.282033 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:04.282036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:04.282038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:04.282120 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:04.282852 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:04.282860 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:04.283246 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 18:41:04.283484 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 18:41:04.283913 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":405,"spinlock_wait_cycles":83456,"thread_start_us":69,"threads_started":1}
I20260807 18:41:04.284144 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:04.261958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:52531 (local address 127.6.138.66:34457)
0807 18:41:04.262370 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:04.262391 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:04.262423 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:41:04.263122 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:04.263128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:04.263465 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:04.263770 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:04.265132 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:04.266104 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:04.267427 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:04.268112 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:04.268325 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:04.278223 (+ 9898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:04.278230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:04.278245 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:04.278278 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:04.280980 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:04.281519 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:04.281524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:04.281528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:04.281661 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:04.282074 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:04.282077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:04.282078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:04.282713 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:04.282855 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:04.283632 (+ 777us) server_negotiation.cc:299] Negotiation successful
0807 18:41:04.283946 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":130,"threads_started":1}
I20260807 18:41:04.284480 30972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:04.262004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:42999 (local address 127.6.138.67:44513)
0807 18:41:04.262421 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:04.262445 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:04.262490 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:41:04.263139 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:04.263144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:04.263466 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:04.263783 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:04.265145 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:04.266107 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:04.267686 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:04.268360 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:04.268513 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:04.278295 (+ 9782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:04.278300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:04.278313 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:04.278346 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:04.280980 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:04.281783 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:04.281789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:04.281795 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:04.281924 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:04.282215 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:04.282218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:04.282219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:04.282713 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:04.282855 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:04.283662 (+ 807us) server_negotiation.cc:299] Negotiation successful
0807 18:41:04.283946 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":88,"threads_started":1}
W20260807 18:41:04.287415 30402 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
W20260807 18:41:04.287626 30400 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
I20260807 18:41:04.287703 30400 leader_election.cc:302] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a3df0b65c64797b1d1d1d89cf996a2; no voters: b0fb355706ad47a7ba33269b6824378d, d5f36c29f6f2487ba435aac2aea35602
I20260807 18:41:04.287968 30965 raft_consensus.cc:2726] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:41:04.834451 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:04.833968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:36421)
0807 18:41:04.834277 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:04.834345 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":87,"threads_started":1}
I20260807 18:41:05.013973 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:05.013495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:37605)
0807 18:41:05.013812 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:05.013878 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":107,"threads_started":1}
I20260807 18:41:05.229835 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:05.229349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:58957)
0807 18:41:05.229629 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:05.229709 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
I20260807 18:41:05.835944 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:05.835422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:45659)
0807 18:41:05.835772 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:05.835838 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":90,"threads_started":1}
I20260807 18:41:06.015362 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:06.014864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:49777)
0807 18:41:06.015183 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:06.015246 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":80,"threads_started":1}
I20260807 18:41:06.231127 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:06.230679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:47747)
0807 18:41:06.230950 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:06.231010 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
I20260807 18:41:06.231714 30983 raft_consensus.cc:482] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:41:06.231825 30983 raft_consensus.cc:504] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:06.232241 30983 leader_election.cc:288] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457), d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513)
W20260807 18:41:06.233091 30402 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
W20260807 18:41:06.233194 30400 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
I20260807 18:41:06.233310 30400 leader_election.cc:302] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a3df0b65c64797b1d1d1d89cf996a2; no voters: b0fb355706ad47a7ba33269b6824378d, d5f36c29f6f2487ba435aac2aea35602
I20260807 18:41:06.233492 30983 raft_consensus.cc:2726] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:41:06.837296 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:06.836830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:54451)
0807 18:41:06.837124 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:06.837196 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":81,"threads_started":1}
I20260807 18:41:07.016765 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:07.016252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:58327)
0807 18:41:07.016602 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:07.016667 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":81,"threads_started":1}
I20260807 18:41:07.232564 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:07.232064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:58377)
0807 18:41:07.232411 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:07.232467 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260807 18:41:07.838774 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:07.838296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:46045)
0807 18:41:07.838613 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:07.838676 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":103,"threads_started":1}
I20260807 18:41:08.018244 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:08.017727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:57839)
0807 18:41:08.018074 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:08.018142 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
I20260807 18:41:08.233780 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:08.233370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:48987)
0807 18:41:08.233633 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:08.233686 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":81,"threads_started":1}
I20260807 18:41:08.839963 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:08.839549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:37177)
0807 18:41:08.839807 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:08.839862 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260807 18:41:09.019672 30994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:09.019207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:56393)
0807 18:41:09.019503 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:09.019568 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":82,"threads_started":1}
I20260807 18:41:09.034858 30995 raft_consensus.cc:482] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:41:09.035020 30995 raft_consensus.cc:504] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:09.035591 30995 leader_election.cc:288] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457), d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513)
W20260807 18:41:09.036397 30400 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
W20260807 18:41:09.036506 30402 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
I20260807 18:41:09.036661 30402 leader_election.cc:302] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a3df0b65c64797b1d1d1d89cf996a2; no voters: b0fb355706ad47a7ba33269b6824378d, d5f36c29f6f2487ba435aac2aea35602
I20260807 18:41:09.036886 30995 raft_consensus.cc:2726] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:41:09.235122 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:09.234621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:59731)
0807 18:41:09.234922 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:09.234998 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
I20260807 18:41:09.841256 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:09.840800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:40621)
0807 18:41:09.841085 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:09.841146 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
I20260807 18:41:10.021214 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:10.020656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:54681)
0807 18:41:10.021046 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:10.021113 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":102,"threads_started":1}
I20260807 18:41:10.236400 31002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:10.235900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:45861)
0807 18:41:10.236222 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:10.236302 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":78,"threads_started":1}
I20260807 18:41:10.842548 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:10.842078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:48227)
0807 18:41:10.842372 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:10.842438 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
I20260807 18:41:10.893126 31004 raft_consensus.cc:482] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:41:10.893272 31004 raft_consensus.cc:504] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:10.893754 31004 leader_election.cc:288] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457), d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513)
W20260807 18:41:10.894568 30400 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
W20260807 18:41:10.894691 30402 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
I20260807 18:41:10.894788 30402 leader_election.cc:302] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a3df0b65c64797b1d1d1d89cf996a2; no voters: b0fb355706ad47a7ba33269b6824378d, d5f36c29f6f2487ba435aac2aea35602
I20260807 18:41:10.894966 31004 raft_consensus.cc:2726] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:41:11.022619 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:11.022031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:32879)
0807 18:41:11.022414 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:11.022478 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":120,"threads_started":1}
I20260807 18:41:11.237741 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:11.237205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:35083)
0807 18:41:11.237519 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:11.237609 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":83,"threads_started":1}
I20260807 18:41:11.843889 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:11.843422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:56911)
0807 18:41:11.843723 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:11.843790 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":82,"threads_started":1}
I20260807 18:41:12.023804 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:12.023384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:44101)
0807 18:41:12.023640 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:12.023696 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
I20260807 18:41:12.239051 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:12.238562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:53741)
0807 18:41:12.238888 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:12.238953 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":109,"threads_started":1}
I20260807 18:41:12.845274 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:12.844847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:41649)
0807 18:41:12.845116 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:12.845176 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260807 18:41:13.025177 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:13.024730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:35021)
0807 18:41:13.025017 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:13.025080 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":82,"threads_started":1}
I20260807 18:41:13.240417 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:13.239877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:58715)
0807 18:41:13.240174 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:13.240238 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":119,"threads_started":1}
I20260807 18:41:13.846549 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:13.846087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:57725)
0807 18:41:13.846363 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:13.846440 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
I20260807 18:41:14.026446 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:14.025981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:55637)
0807 18:41:14.026278 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:14.026337 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":87,"threads_started":1}
I20260807 18:41:14.241812 31023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:14.241337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:39427)
0807 18:41:14.241645 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:14.241712 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":112,"threads_started":1}
I20260807 18:41:14.607090 31024 raft_consensus.cc:482] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:41:14.607260 31024 raft_consensus.cc:504] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0fb355706ad47a7ba33269b6824378d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 34457 } } peers { permanent_uuid: "d5f36c29f6f2487ba435aac2aea35602" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 44513 } } peers { permanent_uuid: "d6a3df0b65c64797b1d1d1d89cf996a2" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 44771 } }
I20260807 18:41:14.607796 31024 leader_election.cc:288] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457), d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513)
W20260807 18:41:14.608680 30400 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b0fb355706ad47a7ba33269b6824378d (127.6.138.66:34457): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
W20260807 18:41:14.608798 30402 leader_election.cc:341] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5f36c29f6f2487ba435aac2aea35602 (127.6.138.67:44513): Not found: Tablet not found: e3bfa3997a3a428a8bd371c0af2b9538
I20260807 18:41:14.608928 30402 leader_election.cc:302] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d6a3df0b65c64797b1d1d1d89cf996a2; no voters: b0fb355706ad47a7ba33269b6824378d, d5f36c29f6f2487ba435aac2aea35602
I20260807 18:41:14.609108 31024 raft_consensus.cc:2726] T e3bfa3997a3a428a8bd371c0af2b9538 P d6a3df0b65c64797b1d1d1d89cf996a2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:41:14.848007 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:14.847464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:43469)
0807 18:41:14.847795 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:14.847861 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":106,"threads_started":1}
I20260807 18:41:15.027792 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:15.027299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:35055)
0807 18:41:15.027599 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:15.027667 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":94,"threads_started":1}
I20260807 18:41:15.243196 31027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:15.242700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:47345)
0807 18:41:15.243040 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:15.243100 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":123,"threads_started":1}
I20260807 18:41:15.849452 31031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:15.848874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:33763)
0807 18:41:15.849286 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:15.849349 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":148,"threads_started":1}
I20260807 18:41:16.029783 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:16.029255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:57943)
0807 18:41:16.029615 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:16.029684 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":94,"threads_started":1}
I20260807 18:41:16.244580 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:16.244001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:58349)
0807 18:41:16.244384 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:16.244449 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":83,"threads_started":1}
I20260807 18:41:16.851011 31035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:16.850425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:57445)
0807 18:41:16.850768 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:16.850839 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260807 18:41:17.031167 31036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:17.030635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:50933)
0807 18:41:17.030973 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:17.031061 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":104,"threads_started":1}
I20260807 18:41:17.245913 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:17.245425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:35575)
0807 18:41:17.245731 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:17.245795 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":110,"threads_started":1}
I20260807 18:41:17.852689 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:17.852118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.65:40897)
0807 18:41:17.852525 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:17.852587 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":197,"threads_started":1}
I20260807 18:41:18.032727 31040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:18.032219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.66:49393)
0807 18:41:18.032568 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:18.032631 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":92,"threads_started":1}
I20260807 18:41:18.233039 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 30331
I20260807 18:41:18.247566 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:18.247015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:37861 (local address 127.6.138.67:55747)
0807 18:41:18.247366 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:18.247434 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:37861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":75,"threads_started":1}
I20260807 18:41:18.255571 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 30577
I20260807 18:41:18.269742 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 30774
2026-08-07T18:41:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19725 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:41:18 dist-test-slave-2d19 krb5kdc[31044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:41:18 dist-test-slave-2d19 krb5kdc[31044](info): set up 2 sockets
Aug 07 18:41:18 dist-test-slave-2d19 krb5kdc[31044](info): commencing operation
krb5kdc: starting...
W20260807 18:41:19.549079 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:41:19 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:41:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:41:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:41:19.685210 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38847
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:38847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:19.801299 31068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:19.801587 31068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:19.801630 31068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:19.805718 31068 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:41:19.805786 31068 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:41:19.805802 31068 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:41:19.805819 31068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:41:19.805837 31068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:19.805903 31068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:19.805936 31068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:41:19.805960 31068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:41:19.811393 31068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:38847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:19.812582 31068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:19.813745 31068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:19.823204 31073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:19.823356 31068 server_base.cc:1034] running on GCE node
W20260807 18:41:19.823477 31078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:19.823514 31074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:19.824126 31068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:19.825341 31068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:19.826563 31068 hybrid_clock.cc:648] HybridClock initialized: now 1786128079826550 us; error 31 us; skew 500 ppm
Aug 07 18:41:19 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128079, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:19.829613 31068 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:41:19.830574 31068 webserver.cc:460] Webserver started at http://127.6.138.126:37135/ using document root <none> and password file <none>
I20260807 18:41:19.831269 31068 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:19.831341 31068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:19.831529 31068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:41:19.833711 31068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "be75824486274c2e8358a42d753ec1de"
format_stamp: "Formatted at 2026-08-07 18:41:19 on dist-test-slave-2d19"
server_key: "b8682c3c67bff05ca7043e5e9926b665"
server_key_iv: "0086aa29ab24c9ce1a76b109b57f9017"
server_key_version: "encryptionkey@0"
I20260807 18:41:19.834257 31068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "be75824486274c2e8358a42d753ec1de"
format_stamp: "Formatted at 2026-08-07 18:41:19 on dist-test-slave-2d19"
server_key: "b8682c3c67bff05ca7043e5e9926b665"
server_key_iv: "0086aa29ab24c9ce1a76b109b57f9017"
server_key_version: "encryptionkey@0"
I20260807 18:41:19.838210 31068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 18:41:19.840840 31088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:19.842216 31068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:19.842381 31068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "be75824486274c2e8358a42d753ec1de"
format_stamp: "Formatted at 2026-08-07 18:41:19 on dist-test-slave-2d19"
server_key: "b8682c3c67bff05ca7043e5e9926b665"
server_key_iv: "0086aa29ab24c9ce1a76b109b57f9017"
server_key_version: "encryptionkey@0"
I20260807 18:41:19.842507 31068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:19.867899 31068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:19.871706 31068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:19.871955 31068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:19.879482 31068 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:38847
I20260807 18:41:19.879781 31172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:38847 every 8 connection(s)
I20260807 18:41:19.880592 31068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:41:19.883114 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 31068
I20260807 18:41:19.883263 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:41:19.883610 6697 external_mini_cluster.cc:1428] Setting key 924206164d95da768d2e1474b30c9c4f
I20260807 18:41:19.883626 31173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:19.890890 31173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Bootstrap starting.
Aug 07 18:41:19 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:19.893855 31173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:19.894785 31173 log.cc:824] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:19.896929 31173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: No bootstrap required, opened a new log
I20260807 18:41:19.898725 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:19.885083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.6.138.126:38847)
0807 18:41:19.885502 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:19.885520 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:19.885559 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:41:19.886534 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:19.886553 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:19.886890 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:19.887181 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:19.888133 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:19.889210 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:19.890406 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:19.890867 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:19.890986 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:19.893901 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:19.893906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:19.893920 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:19.893950 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:19.895729 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:19.896218 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:19.896223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:19.896228 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:19.896402 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:19.896703 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:19.896707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:19.896709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:19.897182 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:19.897395 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:19.897695 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:41:19.897990 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":149,"threads_started":1}
I20260807 18:41:19.901077 31173 raft_consensus.cc:348] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:41:19.901371 31173 raft_consensus.cc:374] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:19.901458 31173 raft_consensus.cc:729] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be75824486274c2e8358a42d753ec1de, State: Initialized, Role: FOLLOWER
I20260807 18:41:19.901873 31173 consensus_queue.cc:260] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:41:19.902029 31173 raft_consensus.cc:388] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:41:19.902117 31173 raft_consensus.cc:482] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:41:19.902221 31173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:19.903169 31173 raft_consensus.cc:504] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:41:19.903524 31173 leader_election.cc:302] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be75824486274c2e8358a42d753ec1de; no voters:
I20260807 18:41:19.903862 31173 leader_election.cc:288] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:41:19.904068 31179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:41:19.904428 31179 raft_consensus.cc:686] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 1 LEADER]: Becoming Leader. State: Replica: be75824486274c2e8358a42d753ec1de, State: Running, Role: LEADER
I20260807 18:41:19.904841 31179 consensus_queue.cc:237] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:41:19.905243 31173 sys_catalog.cc:564] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:41:19.906852 31181 sys_catalog.cc:455] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: SysCatalogTable state changed. Reason: New leader be75824486274c2e8358a42d753ec1de. Latest consensus state: current_term: 1 leader_uuid: "be75824486274c2e8358a42d753ec1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } } }
I20260807 18:41:19.907045 31181 sys_catalog.cc:458] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:19.906852 31180 sys_catalog.cc:455] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be75824486274c2e8358a42d753ec1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } } }
I20260807 18:41:19.907125 31180 sys_catalog.cc:458] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:19.908004 31200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:41:19.910923 31200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:41:19.917166 31200 catalog_manager.cc:1347] Generated new cluster ID: d84cf33a9ef64897a8ddd37bf17b5f1d
I20260807 18:41:19.917282 31200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:41:19.927477 31200 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:41:19.928799 31200 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:41:19.940793 31200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Generated new TSK 0
I20260807 18:41:19.941576 31200 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:41:19.973685 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38847
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:20.092674 31215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:20.092980 31215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:20.093060 31215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:20.097057 31215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:20.097144 31215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:20.097234 31215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:41:20.102651 31215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:20.103772 31215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:20.104914 31215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:20.115604 31226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:20.115599 31224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:20.115680 31215 server_base.cc:1034] running on GCE node
W20260807 18:41:20.115599 31231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:20.116359 31215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:20.117092 31215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:20.118289 31215 hybrid_clock.cc:648] HybridClock initialized: now 1786128080118273 us; error 35 us; skew 500 ppm
Aug 07 18:41:20 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128080, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:20.121240 31215 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:41:20.122192 31215 webserver.cc:460] Webserver started at http://127.6.138.65:40769/ using document root <none> and password file <none>
I20260807 18:41:20.122781 31215 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:20.122879 31215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:20.123322 31215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:41:20.125800 31215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "7ad19809f887452bb6a409877ee471c6"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "97d1513c76c343f72ee184454626fe5e"
server_key_iv: "04132699bbca835323d03b74a495699e"
server_key_version: "encryptionkey@0"
I20260807 18:41:20.126468 31215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "7ad19809f887452bb6a409877ee471c6"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "97d1513c76c343f72ee184454626fe5e"
server_key_iv: "04132699bbca835323d03b74a495699e"
server_key_version: "encryptionkey@0"
I20260807 18:41:20.130836 31215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:41:20.133724 31239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:20.135051 31215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:41:20.135195 31215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "7ad19809f887452bb6a409877ee471c6"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "97d1513c76c343f72ee184454626fe5e"
server_key_iv: "04132699bbca835323d03b74a495699e"
server_key_version: "encryptionkey@0"
I20260807 18:41:20.135316 31215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:20.150992 31215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:20.154284 31215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:20.154456 31215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:20.155016 31215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:20.155800 31215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:20.155846 31215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:20.155892 31215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:20.155908 31215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:20.165598 31215 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:34031
I20260807 18:41:20.165625 31405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:34031 every 8 connection(s)
I20260807 18:41:20.166539 31215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:41:20.171624 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 31215
I20260807 18:41:20.171784 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:41:20.172151 6697 external_mini_cluster.cc:1428] Setting key bdfb7b165ce969dd04cbae6f6c0cd474
Aug 07 18:41:20 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128080, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:20.181982 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.168460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:51775 (local address 127.6.138.126:38847)
0807 18:41:20.168620 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.168625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.169797 (+ 1172us) server_negotiation.cc:407] Connection header received
0807 18:41:20.170687 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.170690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.170736 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.170816 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:20.172185 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.172954 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.174586 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.175024 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.175145 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.178006 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:20.178010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:20.178012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:20.178035 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:20.179671 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:20.180223 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:20.180226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:20.180228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:20.180376 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:20.180735 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:20.180739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:20.180740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:20.180934 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:20.181128 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.181661 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.181801 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:41:20.182807 31408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.168945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:51775)
0807 18:41:20.169648 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:20.169690 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:20.170462 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:20.171022 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:20.171029 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:20.171394 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:20.172027 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.172044 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.173099 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.173102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.174453 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.174468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.175208 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.175211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.175354 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.176009 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:20.176033 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:20.177827 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:20.179814 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:20.179820 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:20.179842 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:20.180110 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:20.180475 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:20.180478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:20.180480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:20.180632 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:20.181128 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:20.181136 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:20.181398 (+ 262us) client_negotiation.cc:769] Sending connection context
0807 18:41:20.181600 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 18:41:20.181940 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:41:20.184106 31406 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38847
I20260807 18:41:20.184405 31406 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:20.184922 31406 heartbeater.cc:511] Master 127.6.138.126:38847 requested a full tablet report, sending...
I20260807 18:41:20.186533 31113 ts_manager.cc:194] Registered new tserver with Master: 7ad19809f887452bb6a409877ee471c6 (127.6.138.65:34031)
I20260807 18:41:20.187608 31113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:51775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:41:20.205876 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38847
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:20.322368 31412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:20.322688 31412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:20.322762 31412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:20.326833 31412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:20.326905 31412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:20.327000 31412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:41:20.332719 31412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:20.333875 31412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:20.334940 31412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:20.345387 31424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:20.345373 31427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:20.345372 31421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:20.345443 31412 server_base.cc:1034] running on GCE node
I20260807 18:41:20.346185 31412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:20.346915 31412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:20.348100 31412 hybrid_clock.cc:648] HybridClock initialized: now 1786128080348079 us; error 34 us; skew 500 ppm
Aug 07 18:41:20 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128080, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:20.351078 31412 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:41:20.352015 31412 webserver.cc:460] Webserver started at http://127.6.138.66:34857/ using document root <none> and password file <none>
I20260807 18:41:20.352720 31412 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:20.352787 31412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:20.353010 31412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:41:20.355108 31412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "3e52ea3713d747e4bc681e4e227097bd"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "fd95cae78186bdd38bb6e1c3a1b3c3bc"
server_key_iv: "bd0fb099e65365efa20e31a1ddb0596c"
server_key_version: "encryptionkey@0"
I20260807 18:41:20.355692 31412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "3e52ea3713d747e4bc681e4e227097bd"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "fd95cae78186bdd38bb6e1c3a1b3c3bc"
server_key_iv: "bd0fb099e65365efa20e31a1ddb0596c"
server_key_version: "encryptionkey@0"
I20260807 18:41:20.359635 31412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:41:20.362483 31437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:20.363824 31412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:41:20.363971 31412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "3e52ea3713d747e4bc681e4e227097bd"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "fd95cae78186bdd38bb6e1c3a1b3c3bc"
server_key_iv: "bd0fb099e65365efa20e31a1ddb0596c"
server_key_version: "encryptionkey@0"
I20260807 18:41:20.364143 31412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:20.383764 31412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:20.386951 31412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:20.387177 31412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:20.387807 31412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:20.388792 31412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:20.388865 31412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:20.388942 31412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:20.388988 31412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:20.398675 31412 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:46749
I20260807 18:41:20.398744 31603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:46749 every 8 connection(s)
I20260807 18:41:20.399652 31412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:41:20.403569 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 31412
I20260807 18:41:20.403705 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:41:20.404011 6697 external_mini_cluster.cc:1428] Setting key d7bfe0cdabac97f9a19ccbe98b99e996
Aug 07 18:41:20 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128080, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:41:20.415975 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.401596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:51541 (local address 127.6.138.126:38847)
0807 18:41:20.401738 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.401742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.402880 (+ 1138us) server_negotiation.cc:407] Connection header received
0807 18:41:20.404155 (+ 1275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.404158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.404208 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.404348 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:20.405838 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.406490 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.408243 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.408657 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.408781 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.411694 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:20.411701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:20.411707 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:20.411742 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:20.413591 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:20.414204 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:20.414210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:20.414212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:20.414311 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:20.414671 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:20.414675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:20.414677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:20.414924 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:20.415046 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.415607 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.415747 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:41:20.416833 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.401988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:51541)
0807 18:41:20.402727 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:20.402770 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:20.403830 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:20.404530 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:20.404538 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:20.404965 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:20.405682 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.405704 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.406635 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.406641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.408120 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.408132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.408819 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.408822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.408966 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.409636 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:20.409652 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:20.411513 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:20.413737 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:20.413742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:20.413756 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:20.414084 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:20.414423 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:20.414426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:20.414427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:20.414570 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:20.415090 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:20.415096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:20.415348 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 18:41:20.415512 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 18:41:20.415779 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":109,"threads_started":1}
I20260807 18:41:20.418076 31604 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38847
I20260807 18:41:20.418426 31604 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
I20260807 18:41:20.419018 31604 heartbeater.cc:511] Master 127.6.138.126:38847 requested a full tablet report, sending...
I20260807 18:41:20.420197 31113 ts_manager.cc:194] Registered new tserver with Master: 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749)
I20260807 18:41:20.420815 31113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:51541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:41:20.436635 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38847
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:20.555368 31611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:20.555687 31611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:20.555765 31611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:20.559840 31611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:20.559945 31611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:20.560043 31611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:41:20.565433 31611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:20.566553 31611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:20.567718 31611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:20.578372 31625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:20.578594 31618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:20.578490 31611 server_base.cc:1034] running on GCE node
W20260807 18:41:20.578438 31623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:20.579037 31611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:20.579783 31611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:20.580955 31611 hybrid_clock.cc:648] HybridClock initialized: now 1786128080580957 us; error 50 us; skew 500 ppm
Aug 07 18:41:20 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128080, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:20.584089 31611 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:41:20.585081 31611 webserver.cc:460] Webserver started at http://127.6.138.67:44249/ using document root <none> and password file <none>
I20260807 18:41:20.585631 31611 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:20.585692 31611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:20.585865 31611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:41:20.587895 31611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "f7570f9b221446cd875c4b0b208d3389"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "55d019242d01b87a2b9e18dbcd790386"
server_key_iv: "8c3a647756383ea41d2fdb3dd9942610"
server_key_version: "encryptionkey@0"
I20260807 18:41:20.588503 31611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "f7570f9b221446cd875c4b0b208d3389"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "55d019242d01b87a2b9e18dbcd790386"
server_key_iv: "8c3a647756383ea41d2fdb3dd9942610"
server_key_version: "encryptionkey@0"
I20260807 18:41:20.592478 31611 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:41:20.595137 31634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:20.596575 31611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:20.596717 31611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "f7570f9b221446cd875c4b0b208d3389"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "55d019242d01b87a2b9e18dbcd790386"
server_key_iv: "8c3a647756383ea41d2fdb3dd9942610"
server_key_version: "encryptionkey@0"
I20260807 18:41:20.596834 31611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:20.617013 31611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:20.620270 31611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:20.620496 31611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:20.621137 31611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:20.622007 31611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:20.622054 31611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:20.622123 31611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:20.622156 31611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:20.632419 31611 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:40983
I20260807 18:41:20.632614 31816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:40983 every 8 connection(s)
I20260807 18:41:20.633574 31611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:41:20.634429 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 31611
I20260807 18:41:20.634584 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:41:20.634886 6697 external_mini_cluster.cc:1428] Setting key 7ffa330e072b925001b432f1e75329ac
Aug 07 18:41:20 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128080, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:20.648146 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.635587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:37339 (local address 127.6.138.126:38847)
0807 18:41:20.635728 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.635731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.637141 (+ 1410us) server_negotiation.cc:407] Connection header received
0807 18:41:20.638081 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.638083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.638130 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.638212 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:20.639465 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.640152 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.641366 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.641594 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.641687 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.644180 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:20.644183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:20.644185 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:20.644211 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:20.645938 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:20.646436 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:20.646439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:20.646440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:20.646523 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:20.646811 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:20.646814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:20.646815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:20.647086 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:20.647169 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.647844 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.647985 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:41:20.649122 31820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.636140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:37339)
0807 18:41:20.636939 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:20.637053 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:20.637875 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:20.638378 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:20.638385 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:20.638757 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:20.639315 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.639330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.640312 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.640315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.641226 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.641235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.641743 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.641746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.641897 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.642477 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:20.642493 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:20.644012 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:20.646043 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:20.646049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:20.646062 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:20.646350 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:20.646586 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:20.646588 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:20.646589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:20.646714 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:20.647187 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:20.647192 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:20.647437 (+ 245us) client_negotiation.cc:769] Sending connection context
0807 18:41:20.647647 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 18:41:20.648011 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":164,"threads_started":1}
I20260807 18:41:20.650308 31817 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38847
I20260807 18:41:20.650554 31817 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:20.651094 31817 heartbeater.cc:511] Master 127.6.138.126:38847 requested a full tablet report, sending...
I20260807 18:41:20.652210 31113 ts_manager.cc:194] Registered new tserver with Master: f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983)
I20260807 18:41:20.652845 31113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:37339
I20260807 18:41:20.661578 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:41:20.671355 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.663736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.6.138.126:38847)
0807 18:41:20.663863 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.663866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.664011 (+ 145us) server_negotiation.cc:407] Connection header received
0807 18:41:20.664120 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.664123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.664163 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.664236 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:20.665000 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.665504 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.666377 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.666594 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.666681 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.668379 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:20.668382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:20.668384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:20.668408 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:20.669802 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:20.670211 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:20.670214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:20.670215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:20.670295 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:20.670540 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:20.670542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:20.670543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:20.670762 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:20.670863 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.671090 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.671201 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:41:20.674623 31113 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:41:20.677706 31113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:41:20.692596 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.686407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34031 (local address 127.0.0.1:33332)
0807 18:41:20.687008 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:20.687060 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:20.687185 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:20.688083 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:20.688089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:20.688112 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:20.688296 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.688306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.689502 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.689508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.690733 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.690745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.691848 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.691853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.692006 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.692063 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 18:41:20.692178 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:41:20.692392 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":94,"threads_started":1}
I20260807 18:41:20.693337 31835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.686571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33332 (local address 127.6.138.65:34031)
0807 18:41:20.687570 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.687577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.687604 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:41:20.687689 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.687695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.687910 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.688076 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:20.688431 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.689387 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.690887 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.691702 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.692833 (+ 1131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.693019 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.693078 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.693184 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":63,"threads_started":1}
I20260807 18:41:20.694136 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.686477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:40983 (local address 127.0.0.1:60706)
0807 18:41:20.687464 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:20.687479 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:20.687581 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:20.688245 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:20.688247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:20.688314 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:20.688430 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.688437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.690045 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.690049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.691809 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.691824 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.693640 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.693645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.693786 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.693821 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:41:20.693905 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 18:41:20.693994 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"mutex_wait_us":3,"thread_start_us":115,"threads_started":1}
I20260807 18:41:20.694703 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.686887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.6.138.67:40983)
0807 18:41:20.687320 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.687334 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.687681 (+ 347us) server_negotiation.cc:407] Connection header received
0807 18:41:20.687755 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.687759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.687909 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.688076 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:20.688563 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.689718 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.692056 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.693268 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.693390 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.693546 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.694363 (+ 817us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.694504 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":131,"threads_started":1}
I20260807 18:41:20.696198 31838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.687173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:46749 (local address 127.0.0.1:60314)
0807 18:41:20.687953 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:20.687964 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:20.688078 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:20.688862 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:20.688865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:20.688884 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:20.689004 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.689011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.690329 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.690332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.691977 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.691990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.695783 (+ 3793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.695785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.695866 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.695884 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:41:20.695988 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:41:20.696073 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":254,"thread_start_us":102,"threads_started":1}
I20260807 18:41:20.696286 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.687575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.6.138.66:46749)
0807 18:41:20.688356 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.688371 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.688398 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:41:20.688473 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.688477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.688709 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.688864 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:20.689129 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.690184 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.692158 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.695489 (+ 3331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.695610 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.695724 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.696024 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.696125 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":112,"threads_started":1}
I20260807 18:41:20.696978 31309 tablet_service.cc:1467] Processing CreateTablet for tablet b062014048ed4f978a754b30f44e3662 (DEFAULT_TABLE table=test-table [id=42fe006a381f459089ccab799f5b0c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:20.698218 31309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b062014048ed4f978a754b30f44e3662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:20.698676 31710 tablet_service.cc:1467] Processing CreateTablet for tablet b062014048ed4f978a754b30f44e3662 (DEFAULT_TABLE table=test-table [id=42fe006a381f459089ccab799f5b0c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:20.699072 31509 tablet_service.cc:1467] Processing CreateTablet for tablet b062014048ed4f978a754b30f44e3662 (DEFAULT_TABLE table=test-table [id=42fe006a381f459089ccab799f5b0c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:20.700148 31710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b062014048ed4f978a754b30f44e3662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:20.700341 31509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b062014048ed4f978a754b30f44e3662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:20.706610 31841 tablet_bootstrap.cc:492] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Bootstrap starting.
I20260807 18:41:20.708168 31843 tablet_bootstrap.cc:492] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: Bootstrap starting.
I20260807 18:41:20.709223 31841 tablet_bootstrap.cc:654] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:20.709990 31841 log.cc:824] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:20.710779 31843 tablet_bootstrap.cc:654] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:20.710772 31844 tablet_bootstrap.cc:492] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: Bootstrap starting.
I20260807 18:41:20.711642 31843 log.cc:824] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:20.712376 31841 tablet_bootstrap.cc:492] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: No bootstrap required, opened a new log
I20260807 18:41:20.712612 31841 ts_tablet_manager.cc:1397] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260807 18:41:20.713398 31844 tablet_bootstrap.cc:654] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:20.713934 31843 tablet_bootstrap.cc:492] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: No bootstrap required, opened a new log
I20260807 18:41:20.714133 31843 ts_tablet_manager.cc:1397] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:41:20.714244 31844 log.cc:824] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:20.716351 31844 tablet_bootstrap.cc:492] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: No bootstrap required, opened a new log
I20260807 18:41:20.716538 31844 ts_tablet_manager.cc:1397] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:41:20.716722 31841 raft_consensus.cc:348] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:20.717020 31841 raft_consensus.cc:374] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:20.717077 31841 raft_consensus.cc:729] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ad19809f887452bb6a409877ee471c6, State: Initialized, Role: FOLLOWER
I20260807 18:41:20.717561 31841 consensus_queue.cc:260] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:20.718035 31843 raft_consensus.cc:348] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:20.718312 31843 raft_consensus.cc:374] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:20.718384 31843 raft_consensus.cc:729] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7570f9b221446cd875c4b0b208d3389, State: Initialized, Role: FOLLOWER
I20260807 18:41:20.718565 31841 ts_tablet_manager.cc:1428] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260807 18:41:20.718822 31843 consensus_queue.cc:260] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:20.718969 31406 heartbeater.cc:503] Master 127.6.138.126:38847 was elected leader, sending a full tablet report...
I20260807 18:41:20.719501 31817 heartbeater.cc:503] Master 127.6.138.126:38847 was elected leader, sending a full tablet report...
I20260807 18:41:20.719847 31843 ts_tablet_manager.cc:1428] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:41:20.720422 31844 raft_consensus.cc:348] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:20.720727 31844 raft_consensus.cc:374] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:20.720791 31844 raft_consensus.cc:729] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e52ea3713d747e4bc681e4e227097bd, State: Initialized, Role: FOLLOWER
I20260807 18:41:20.721221 31844 consensus_queue.cc:260] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:20.721920 31604 heartbeater.cc:503] Master 127.6.138.126:38847 was elected leader, sending a full tablet report...
I20260807 18:41:20.722160 31844 ts_tablet_manager.cc:1428] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:41:20.848043 31850 raft_consensus.cc:482] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:41:20.848292 31850 raft_consensus.cc:504] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:20.849489 31850 leader_election.cc:288] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983), 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749)
I20260807 18:41:20.853736 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.849851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:55735 (local address 127.6.138.67:40983)
0807 18:41:20.850006 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.850010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.850351 (+ 341us) server_negotiation.cc:407] Connection header received
0807 18:41:20.850510 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.850512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.850555 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.850656 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:20.850994 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.851478 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.852736 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.853146 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.853243 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.853327 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.853523 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.853607 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:41:20.853772 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.849918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:38463 (local address 127.6.138.66:46749)
0807 18:41:20.850057 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.850062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.850284 (+ 222us) server_negotiation.cc:407] Connection header received
0807 18:41:20.850461 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.850466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.850531 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.850594 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:20.850951 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.851473 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.852779 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.853176 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.853316 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.853372 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.853580 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.853662 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:41:20.853763 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.849790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:40983 (local address 127.6.138.65:55735)
0807 18:41:20.850208 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:20.850220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:20.850365 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:20.850693 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:20.850705 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:20.850723 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:20.850849 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.850854 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.851596 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.851600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.852606 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.852616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.853276 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.853280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.853374 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.853404 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:41:20.853478 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:41:20.853573 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":121,"thread_start_us":162,"threads_started":1}
I20260807 18:41:20.853771 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.849789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:46749 (local address 127.6.138.65:38463)
0807 18:41:20.850171 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:20.850184 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:20.850324 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:20.850641 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:20.850647 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:20.850663 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:20.850817 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.850823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.851621 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.851624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.852622 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:20.852632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.853329 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.853333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:20.853429 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.853446 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 18:41:20.853545 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 18:41:20.853620 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":38,"thread_start_us":148,"threads_started":1}
I20260807 18:41:20.854533 31737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b062014048ed4f978a754b30f44e3662" candidate_uuid: "7ad19809f887452bb6a409877ee471c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7570f9b221446cd875c4b0b208d3389" is_pre_election: true
I20260807 18:41:20.854535 31541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b062014048ed4f978a754b30f44e3662" candidate_uuid: "7ad19809f887452bb6a409877ee471c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e52ea3713d747e4bc681e4e227097bd" is_pre_election: true
I20260807 18:41:20.854842 31541 raft_consensus.cc:2454] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ad19809f887452bb6a409877ee471c6 in term 0.
I20260807 18:41:20.854847 31737 raft_consensus.cc:2454] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ad19809f887452bb6a409877ee471c6 in term 0.
I20260807 18:41:20.855459 31240 leader_election.cc:302] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3e52ea3713d747e4bc681e4e227097bd, 7ad19809f887452bb6a409877ee471c6, f7570f9b221446cd875c4b0b208d3389; no voters:
I20260807 18:41:20.855868 31850 raft_consensus.cc:2781] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:41:20.855995 31850 raft_consensus.cc:482] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:41:20.856066 31850 raft_consensus.cc:3037] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:20.857203 31850 raft_consensus.cc:504] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:20.857527 31850 leader_election.cc:288] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 1 election: Requested vote from peers f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983), 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749)
I20260807 18:41:20.857935 31737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b062014048ed4f978a754b30f44e3662" candidate_uuid: "7ad19809f887452bb6a409877ee471c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7570f9b221446cd875c4b0b208d3389"
I20260807 18:41:20.857936 31541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b062014048ed4f978a754b30f44e3662" candidate_uuid: "7ad19809f887452bb6a409877ee471c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e52ea3713d747e4bc681e4e227097bd"
I20260807 18:41:20.858071 31541 raft_consensus.cc:3037] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:20.858095 31737 raft_consensus.cc:3037] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:20.859460 31737 raft_consensus.cc:2454] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ad19809f887452bb6a409877ee471c6 in term 1.
I20260807 18:41:20.859460 31541 raft_consensus.cc:2454] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ad19809f887452bb6a409877ee471c6 in term 1.
I20260807 18:41:20.859793 31242 leader_election.cc:302] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ad19809f887452bb6a409877ee471c6, f7570f9b221446cd875c4b0b208d3389; no voters:
I20260807 18:41:20.860013 31850 raft_consensus.cc:2781] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:41:20.860319 31850 raft_consensus.cc:686] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 1 LEADER]: Becoming Leader. State: Replica: 7ad19809f887452bb6a409877ee471c6, State: Running, Role: LEADER
I20260807 18:41:20.860620 31850 consensus_queue.cc:237] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:20.863951 31108 catalog_manager.cc:5495] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ad19809f887452bb6a409877ee471c6 (127.6.138.65). New cstate: current_term: 1 leader_uuid: "7ad19809f887452bb6a409877ee471c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } health_report { overall_health: UNKNOWN } } }
W20260807 18:41:20.884645 31819 tablet.cc:2369] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:41:20.892551 31835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.888480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33348 (local address 127.6.138.65:34031)
0807 18:41:20.888641 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:20.888646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:20.888663 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:41:20.888789 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:20.888792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:20.888859 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:20.888967 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:41:20.889215 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.890011 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.890866 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:20.891212 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:20.891315 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:20.891594 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:41:20.892115 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:41:20.892233 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:20.892344 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 18:41:20.892414 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:41:20.899040 31541 raft_consensus.cc:1261] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [term 1 FOLLOWER]: Refusing update from remote peer 7ad19809f887452bb6a409877ee471c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:41:20.899065 31737 raft_consensus.cc:1261] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [term 1 FOLLOWER]: Refusing update from remote peer 7ad19809f887452bb6a409877ee471c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:41:20.899811 31857 consensus_queue.cc:1035] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:41:20.899992 31850 consensus_queue.cc:1035] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:41:20.900830 31605 tablet.cc:2369] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:41:20.908146 31865 mvcc.cc:205] Tried to move back new op lower bound from 7315980619354648576 to 7315980619207680000. Current Snapshot: MvccSnapshot[applied={T|T < 7315980619354648576}]
I20260807 18:41:20.917639 31108 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:41:20.917850 31108 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:41:20.920677 31108 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Sending DeleteTablet for 3 replicas of tablet b062014048ed4f978a754b30f44e3662
I20260807 18:41:20.921631 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 31215
I20260807 18:41:20.921545 31710 tablet_service.cc:1514] Processing DeleteTablet for tablet b062014048ed4f978a754b30f44e3662 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:20 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:60706
I20260807 18:41:20.921770 31309 tablet_service.cc:1514] Processing DeleteTablet for tablet b062014048ed4f978a754b30f44e3662 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:20 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:33332
I20260807 18:41:20.922363 31509 tablet_service.cc:1514] Processing DeleteTablet for tablet b062014048ed4f978a754b30f44e3662 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:20 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:60314
I20260807 18:41:20.924919 31878 tablet_replica.cc:330] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: stopping tablet replica
I20260807 18:41:20.924940 31881 tablet_replica.cc:330] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: stopping tablet replica
I20260807 18:41:20.925318 31878 raft_consensus.cc:2229] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:41:20.925318 31881 raft_consensus.cc:2229] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:41:20.925695 31881 raft_consensus.cc:2258] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:20.925695 31878 raft_consensus.cc:2258] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:20.929713 31878 ts_tablet_manager.cc:1905] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:20.932495 31881 ts_tablet_manager.cc:1905] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:20.933144 31878 ts_tablet_manager.cc:1918] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:20.933248 31878 log.cc:1196] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/b062014048ed4f978a754b30f44e3662
I20260807 18:41:20.933590 31878 ts_tablet_manager.cc:1939] T b062014048ed4f978a754b30f44e3662 P f7570f9b221446cd875c4b0b208d3389: Deleting consensus metadata
I20260807 18:41:20.935133 31093 catalog_manager.cc:4841] TS f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983): tablet b062014048ed4f978a754b30f44e3662 (table test-table [id=42fe006a381f459089ccab799f5b0c67]) successfully deleted
I20260807 18:41:20.935427 31881 ts_tablet_manager.cc:1918] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:20.935563 31881 log.cc:1196] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/b062014048ed4f978a754b30f44e3662
I20260807 18:41:20.935861 31881 ts_tablet_manager.cc:1939] T b062014048ed4f978a754b30f44e3662 P 3e52ea3713d747e4bc681e4e227097bd: Deleting consensus metadata
I20260807 18:41:20.937105 31089 catalog_manager.cc:4841] TS 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749): tablet b062014048ed4f978a754b30f44e3662 (table test-table [id=42fe006a381f459089ccab799f5b0c67]) successfully deleted
W20260807 18:41:20.940913 31089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:34031 (error 108)
I20260807 18:41:20.941013 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 31412
I20260807 18:41:20.947161 31838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:20.946593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34031 (local address 127.0.0.1:33350)
0807 18:41:20.946883 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:20.947022 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
W20260807 18:41:20.947436 31089 catalog_manager.cc:4584] TS 7ad19809f887452bb6a409877ee471c6 (127.6.138.65:34031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b062014048ed4f978a754b30f44e3662: Network error: Client connection negotiation failed: client connection to 127.6.138.65:34031: connect: Connection refused (error 111)
I20260807 18:41:20.954233 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 31611
I20260807 18:41:20.969693 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 31068
I20260807 18:41:40.983677 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38847
--webserver_interface=127.6.138.126
--webserver_port=37135
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:38847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:41:41.103749 31883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:41.104024 31883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:41.104067 31883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:41.108085 31883 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:41:41.108146 31883 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:41:41.108163 31883 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:41:41.108177 31883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:41:41.108196 31883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:41.108213 31883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:41.108229 31883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:41:41.108244 31883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:41:41.113596 31883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:38847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=37135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:41.114630 31883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:41.115849 31883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:41:41.130736 31883 server_base.cc:1034] running on GCE node
W20260807 18:41:41.130647 31896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:41.130685 31895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:41.130663 31900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:41.131650 31883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:41.132988 31883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:41.134169 31883 hybrid_clock.cc:648] HybridClock initialized: now 1786128101134132 us; error 53 us; skew 500 ppm
Aug 07 18:41:41 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128101, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:41.142575 31883 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:41:41.143607 31883 webserver.cc:460] Webserver started at http://127.6.138.126:37135/ using document root <none> and password file <none>
I20260807 18:41:41.144328 31883 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:41.144455 31883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:41.149017 31883 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:41.151368 31908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:41.152951 31883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:41:41.153119 31883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "be75824486274c2e8358a42d753ec1de"
format_stamp: "Formatted at 2026-08-07 18:41:19 on dist-test-slave-2d19"
server_key: "b8682c3c67bff05ca7043e5e9926b665"
server_key_iv: "0086aa29ab24c9ce1a76b109b57f9017"
server_key_version: "encryptionkey@0"
I20260807 18:41:41.153790 31883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:41.176352 31883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:41.179672 31883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:41.179894 31883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:41.188112 31883 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:38847
I20260807 18:41:41.188468 31989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:38847 every 8 connection(s)
I20260807 18:41:41.189322 31883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:41:41.192909 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 31883
I20260807 18:41:41.193394 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:34031
--local_ip_for_outbound_sockets=127.6.138.65
--tserver_master_addrs=127.6.138.126:38847
--webserver_port=40769
--webserver_interface=127.6.138.65
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:41:41.194366 31990 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:41:41.196694 31990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Bootstrap starting.
I20260807 18:41:41.224017 31990 log.cc:824] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:41.236238 31990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:41:41.236794 31990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Bootstrap complete.
I20260807 18:41:41.242903 31990 raft_consensus.cc:348] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:41:41.243492 31990 raft_consensus.cc:729] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: be75824486274c2e8358a42d753ec1de, State: Initialized, Role: FOLLOWER
I20260807 18:41:41.244136 31990 consensus_queue.cc:260] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:41:41.244341 31990 raft_consensus.cc:388] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:41:41.244454 31990 raft_consensus.cc:482] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:41:41.244570 31990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:41:41.247373 31990 raft_consensus.cc:504] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:41:41.247792 31990 leader_election.cc:302] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be75824486274c2e8358a42d753ec1de; no voters:
I20260807 18:41:41.248212 31990 leader_election.cc:288] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:41:41.248373 31997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:41:41.248652 31997 raft_consensus.cc:686] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 2 LEADER]: Becoming Leader. State: Replica: be75824486274c2e8358a42d753ec1de, State: Running, Role: LEADER
I20260807 18:41:41.248984 31997 consensus_queue.cc:237] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:41:41.249650 31990 sys_catalog.cc:564] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:41:41.250576 31999 sys_catalog.cc:455] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: SysCatalogTable state changed. Reason: New leader be75824486274c2e8358a42d753ec1de. Latest consensus state: current_term: 2 leader_uuid: "be75824486274c2e8358a42d753ec1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } } }
I20260807 18:41:41.250708 31999 sys_catalog.cc:458] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:41.251459 31998 sys_catalog.cc:455] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "be75824486274c2e8358a42d753ec1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } } }
I20260807 18:41:41.251565 31998 sys_catalog.cc:458] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: This master's current role is: LEADER
I20260807 18:41:41.251708 32009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:41:41.256865 32009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:41:41.257596 32009 catalog_manager.cc:1259] Loaded cluster ID: d84cf33a9ef64897a8ddd37bf17b5f1d
I20260807 18:41:41.257661 32009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:41:41.260195 32009 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:41:41.260958 32009 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Loaded TSK: 0
I20260807 18:41:41.281708 32009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Generated new TSK 1
I20260807 18:41:41.282006 32009 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:41:41.345247 31995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:41.345675 31995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:41.345757 31995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:41.354291 31995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:41.354403 31995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:41.354585 31995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:41:41.364697 31995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:34031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=40769
--tserver_master_addrs=127.6.138.126:38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:41.366588 31995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:41.367828 31995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:41.381072 32035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:41.382606 32032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:41.384727 32037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:41.389699 31995 server_base.cc:1034] running on GCE node
I20260807 18:41:41.390319 31995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:41.391244 31995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:41.392519 31995 hybrid_clock.cc:648] HybridClock initialized: now 1786128101392435 us; error 99 us; skew 500 ppm
Aug 07 18:41:41 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128101, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:41.396824 31995 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:41:41.398137 31995 webserver.cc:460] Webserver started at http://127.6.138.65:40769/ using document root <none> and password file <none>
I20260807 18:41:41.398960 31995 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:41.399092 31995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:41.405133 31995 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:41:41.408239 32053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:41.409945 31995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:41:41.410140 31995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "7ad19809f887452bb6a409877ee471c6"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "97d1513c76c343f72ee184454626fe5e"
server_key_iv: "04132699bbca835323d03b74a495699e"
server_key_version: "encryptionkey@0"
I20260807 18:41:41.411060 31995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:41.445298 31995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:41.448946 31995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:41.449195 31995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:41.449994 31995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:41.451287 32061 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 18:41:41.455724 31995 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:41:41.455797 31995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260807 18:41:41.455884 31995 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:41:41.457808 31995 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:41:41.457877 31995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 18:41:41.458138 32061 tablet_bootstrap.cc:492] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Bootstrap starting.
I20260807 18:41:41.469803 31995 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:34031
I20260807 18:41:41.469877 32223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:34031 every 8 connection(s)
I20260807 18:41:41.471177 31995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:41:41.477124 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 31995
I20260807 18:41:41.477772 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:46749
--local_ip_for_outbound_sockets=127.6.138.66
--tserver_master_addrs=127.6.138.126:38847
--webserver_port=34857
--webserver_interface=127.6.138.66
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:41:41.481388 32061 log.cc:824] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:41:41 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128101, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:41.490401 32227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.473613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:53809 (local address 127.6.138.126:38847)
0807 18:41:41.474003 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:41.474021 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:41.474480 (+ 459us) server_negotiation.cc:407] Connection header received
0807 18:41:41.475769 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:41.475781 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:41.476118 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:41.476531 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:41.477871 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.478972 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.480667 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.481430 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.481628 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.484639 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:41.484646 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:41.484661 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:41.484695 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:41.486738 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.487409 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.487417 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.487423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.487583 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.487933 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.487937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.487942 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.488413 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:41.488576 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:41.489153 (+ 577us) server_negotiation.cc:299] Negotiation successful
0807 18:41:41.489426 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":133,"threads_started":1}
I20260807 18:41:41.490417 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.473624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:53809)
0807 18:41:41.474290 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:41.474333 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:41.475298 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:41.476594 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:41.476603 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:41.477089 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:41.477665 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.477682 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.479163 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.479166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.480504 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.480521 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.481702 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.481705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.481923 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.482708 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:41.482726 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:41.484396 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:41.486881 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:41.486888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:41.486903 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:41.487245 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:41.487694 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:41.487696 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:41.487697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:41.487827 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:41.488546 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:41.488553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:41.488812 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 18:41:41.489024 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 18:41:41.489399 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":168,"threads_started":1}
I20260807 18:41:41.492801 32061 tablet_bootstrap.cc:492] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:41:41.493160 32224 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38847
I20260807 18:41:41.493317 32061 tablet_bootstrap.cc:492] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Bootstrap complete.
I20260807 18:41:41.493428 32224 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:41.493989 32224 heartbeater.cc:511] Master 127.6.138.126:38847 requested a full tablet report, sending...
I20260807 18:41:41.495025 32061 ts_tablet_manager.cc:1397] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Time spent bootstrapping tablet: real 0.037s user 0.030s sys 0.004s
I20260807 18:41:41.496706 31930 ts_manager.cc:194] Registered new tserver with Master: 7ad19809f887452bb6a409877ee471c6 (127.6.138.65:34031)
I20260807 18:41:41.498193 32061 raft_consensus.cc:348] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:41.498764 32061 raft_consensus.cc:729] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ad19809f887452bb6a409877ee471c6, State: Initialized, Role: FOLLOWER
I20260807 18:41:41.499290 32061 consensus_queue.cc:260] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } }
I20260807 18:41:41.500451 32061 ts_tablet_manager.cc:1428] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260807 18:41:41.501238 31930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:53809
I20260807 18:41:41.504138 32224 heartbeater.cc:503] Master 127.6.138.126:38847 was elected leader, sending a full tablet report...
Aug 07 18:41:41 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128101, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.65@KRBTEST.COM
I20260807 18:41:41.511649 32235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.500127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.6.138.65:34031)
0807 18:41:41.501550 (+ 1423us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:41.501562 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:41.501585 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:41:41.501657 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:41.501662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:41.501886 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:41.502016 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:41.502781 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.503505 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.504580 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.505047 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.505163 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.507869 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:41.507888 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:41.507902 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:41.507930 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:41.509609 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.510151 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.510157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.510158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.510242 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.510553 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.510557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.510558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.510811 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:41.510976 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:41.511315 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 18:41:41.511485 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1323,"thread_start_us":81,"threads_started":1}
I20260807 18:41:41.511649 32234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.499922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34031 (local address 127.0.0.1:45006)
0807 18:41:41.501061 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:41.501118 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:41.501268 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:41.502085 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:41.502093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:41.502532 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:41.502647 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.502653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.503632 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.503636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.504367 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.504378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.505176 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.505180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.505311 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.506081 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:41.506096 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:41.507721 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:41.509746 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:41.509757 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:41.509764 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:41.510017 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:41.510345 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:41.510349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:41.510351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:41.510446 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:41.510937 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:41.510943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:41.511081 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 18:41:41.511230 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 18:41:41.511456 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":138,"threads_started":1}
I20260807 18:41:41.513469 32122 tablet_service.cc:1514] Processing DeleteTablet for tablet b062014048ed4f978a754b30f44e3662 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:20 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45006
I20260807 18:41:41.514547 32238 tablet_replica.cc:330] stopping tablet replica
I20260807 18:41:41.514899 32238 raft_consensus.cc:2229] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:41:41.515173 32238 raft_consensus.cc:2258] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:41.516295 32238 ts_tablet_manager.cc:1905] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:41.519935 32238 ts_tablet_manager.cc:1918] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:41.520025 32238 log.cc:1196] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal/wals/b062014048ed4f978a754b30f44e3662
I20260807 18:41:41.520385 32238 ts_tablet_manager.cc:1939] T b062014048ed4f978a754b30f44e3662 P 7ad19809f887452bb6a409877ee471c6: Deleting consensus metadata
I20260807 18:41:41.522235 31909 catalog_manager.cc:4841] TS 7ad19809f887452bb6a409877ee471c6 (127.6.138.65:34031): tablet b062014048ed4f978a754b30f44e3662 (table test-table [id=42fe006a381f459089ccab799f5b0c67]) successfully deleted
W20260807 18:41:41.608882 32231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:41.609174 32231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:41.609220 32231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:41.613272 32231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:41.613335 32231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:41.613400 32231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:41:41.618963 32231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:46749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=34857
--tserver_master_addrs=127.6.138.126:38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:41.620268 32231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:41.621424 32231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:41.631376 32246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:41.631417 32231 server_base.cc:1034] running on GCE node
W20260807 18:41:41.631376 32249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:41.631412 32247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:41.632099 32231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:41.632838 32231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:41.633997 32231 hybrid_clock.cc:648] HybridClock initialized: now 1786128101633982 us; error 36 us; skew 500 ppm
Aug 07 18:41:41 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128101, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:41.636929 32231 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:41:41.637861 32231 webserver.cc:460] Webserver started at http://127.6.138.66:34857/ using document root <none> and password file <none>
I20260807 18:41:41.638403 32231 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:41.638477 32231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:41.642515 32231 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:41:41.644783 32260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:41.646122 32231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:41.646265 32231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "3e52ea3713d747e4bc681e4e227097bd"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "fd95cae78186bdd38bb6e1c3a1b3c3bc"
server_key_iv: "bd0fb099e65365efa20e31a1ddb0596c"
server_key_version: "encryptionkey@0"
I20260807 18:41:41.646831 32231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:41.663064 32231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:41.666255 32231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:41.666466 32231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:41.667078 32231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:41.667941 32231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:41.668015 32231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:41.668093 32231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:41.668138 32231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:41.678339 32231 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:46749
I20260807 18:41:41.678367 32440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:46749 every 8 connection(s)
I20260807 18:41:41.679365 32231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:41:41.687932 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 32231
I20260807 18:41:41.688526 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:40983
--local_ip_for_outbound_sockets=127.6.138.67
--tserver_master_addrs=127.6.138.126:38847
--webserver_port=44249
--webserver_interface=127.6.138.67
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:41:41 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128101, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:41.694314 32227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.681342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:50085 (local address 127.6.138.126:38847)
0807 18:41:41.681480 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:41.681483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:41.682498 (+ 1015us) server_negotiation.cc:407] Connection header received
0807 18:41:41.683435 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:41.683437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:41.683489 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:41.683564 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:41.684844 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.685404 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.686525 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.686812 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.686896 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.689879 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:41.689882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:41.689884 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:41.689909 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:41.691915 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.692630 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.692633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.692634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.692712 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.693148 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.693150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.693151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.693331 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:41.693406 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:41.694064 (+ 658us) server_negotiation.cc:299] Negotiation successful
0807 18:41:41.694188 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:41:41.695758 32443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.681656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:50085)
0807 18:41:41.682313 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:41.682354 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:41.683246 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:41.683776 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:41.683783 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:41.684146 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:41.684702 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.684717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.685515 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.685517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.686395 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.686404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.686949 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.686952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.687093 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.687694 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:41.687710 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:41.689696 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:41.692056 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:41.692061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:41.692075 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:41.692518 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:41.692811 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:41.692817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:41.692822 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:41.693033 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:41.693439 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:41.693446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:41.693790 (+ 344us) client_negotiation.cc:769] Sending connection context
0807 18:41:41.694014 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 18:41:41.694446 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":114,"threads_started":1}
I20260807 18:41:41.697180 32441 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38847
I20260807 18:41:41.697445 32441 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:41.697984 32441 heartbeater.cc:511] Master 127.6.138.126:38847 requested a full tablet report, sending...
I20260807 18:41:41.699132 31930 ts_manager.cc:194] Registered new tserver with Master: 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749)
I20260807 18:41:41.699735 31930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:50085
W20260807 18:41:41.811358 32445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:41:41.811666 32445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:41:41.811710 32445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:41:41.815960 32445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:41:41.816053 32445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:41:41.816115 32445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:41:41.821632 32445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:40983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=44249
--tserver_master_addrs=127.6.138.126:38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:41:41.822832 32445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:41:41.824168 32445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:41:41.834798 32456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:41.834842 32455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:41:41.834798 32461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:41:41.835290 32445 server_base.cc:1034] running on GCE node
I20260807 18:41:41.835737 32445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:41:41.836512 32445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:41:41.837710 32445 hybrid_clock.cc:648] HybridClock initialized: now 1786128101837688 us; error 32 us; skew 500 ppm
Aug 07 18:41:41 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128101, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:41:41.840858 32445 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:41:41.841814 32445 webserver.cc:460] Webserver started at http://127.6.138.67:44249/ using document root <none> and password file <none>
I20260807 18:41:41.842396 32445 fs_manager.cc:359] Metadata directory not provided
I20260807 18:41:41.842478 32445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:41:41.846650 32445 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:41:41.849025 32470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:41.850636 32445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:41:41.850822 32445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "f7570f9b221446cd875c4b0b208d3389"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "55d019242d01b87a2b9e18dbcd790386"
server_key_iv: "8c3a647756383ea41d2fdb3dd9942610"
server_key_version: "encryptionkey@0"
I20260807 18:41:41.851575 32445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:41:41.896184 32445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:41:41.900022 32445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:41:41.900239 32445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:41:41.900955 32445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:41:41.901867 32445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:41:41.901959 32445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:41.902056 32445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:41:41.902108 32445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:41:41.911947 32445 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:40983
I20260807 18:41:41.911962 32663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:40983 every 8 connection(s)
I20260807 18:41:41.913116 32445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:41:41.918516 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 32445
Aug 07 18:41:41 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128101, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:41:41.927673 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.919269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50572 (local address 127.6.138.126:38847)
0807 18:41:41.919635 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:41.919640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:41.919654 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:41:41.919811 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:41.919814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:41.919865 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:41.919960 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:41.920804 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.921391 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.922464 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.922707 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.922817 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.924233 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:41.924237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:41.924239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:41.924297 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:41.925908 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.926345 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.926348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.926350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.926449 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.926735 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.926738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.926740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.926931 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:41.927210 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:41.927334 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 18:41:41.927464 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":278,"thread_start_us":104,"threads_started":1}
I20260807 18:41:41.928020 32227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.915004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:43453 (local address 127.6.138.126:38847)
0807 18:41:41.915163 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:41.915168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:41.916627 (+ 1459us) server_negotiation.cc:407] Connection header received
0807 18:41:41.917521 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:41.917523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:41.917572 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:41.917672 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:41.919018 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.919698 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.920986 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.921215 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.921360 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.924029 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:41.924032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:41.924034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:41.924059 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:41.925777 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.926352 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.926356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.926357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.926476 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.926837 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.926840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.926842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.927038 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:41.927151 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:41.927743 (+ 592us) server_negotiation.cc:299] Negotiation successful
0807 18:41:41.927860 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":57}
I20260807 18:41:41.929252 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.915577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:43453)
0807 18:41:41.916464 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:41.916511 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:41.917310 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:41.917860 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:41.917868 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:41.918266 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:41:41.918856 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.918873 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.919843 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.919847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.920850 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.920860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.921397 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.921404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.921600 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.922288 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:41:41.922305 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:41:41.923839 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:41:41.925940 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:41.925947 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:41.925963 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:41.926240 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:41.926575 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:41:41.926578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:41:41.926580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:41:41.926732 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:41:41.927154 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:41:41.927164 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:41:41.927463 (+ 299us) client_negotiation.cc:769] Sending connection context
0807 18:41:41.927693 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 18:41:41.928044 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":121,"threads_started":1}
I20260807 18:41:41.930402 32665 heartbeater.cc:347] Connected to a master server at 127.6.138.126:38847
I20260807 18:41:41.930680 32665 heartbeater.cc:464] Registering TS with master...
I20260807 18:41:41.931191 32665 heartbeater.cc:511] Master 127.6.138.126:38847 requested a full tablet report, sending...
I20260807 18:41:41.932354 31930 ts_manager.cc:194] Registered new tserver with Master: f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983)
I20260807 18:41:41.932973 31930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:43453
I20260807 18:41:41.941705 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:41:41.946086 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.942339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50584 (local address 127.6.138.126:38847)
0807 18:41:41.942468 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:41.942471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:41.942650 (+ 179us) server_negotiation.cc:407] Connection header received
0807 18:41:41.942798 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:41.942800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:41.942899 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:41.942986 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:41:41.943202 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.943724 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.944902 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.945153 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.945253 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.945628 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:41:41.945769 (+ 141us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 18:41:41.945985 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50584: token expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 18:41:41.946187 32671 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50584: token expired
W20260807 18:41:41.946718 31825 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.138.126:38847)
I20260807 18:41:41.955272 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.947156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50588 (local address 127.6.138.126:38847)
0807 18:41:41.947267 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:41.947271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:41.947289 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:41:41.947342 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:41.947345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:41.947385 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:41.947486 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:41:41.948478 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.948971 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.949783 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.949982 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.950058 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.951739 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:41:41.951741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:41:41.951743 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:41:41.951766 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:41:41.953507 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.953941 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.953944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.953945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.954033 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:41:41.954341 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:41:41.954343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:41:41.954345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:41:41.954569 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:41:41.954683 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:41.955026 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 18:41:41.955144 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 18:41:41.958130 31930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:41:41.961859 31930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:41:41.970554 32122 tablet_service.cc:1467] Processing CreateTablet for tablet 05e2f618ab1b4180aa6e70b888b9c5be (DEFAULT_TABLE table=test-table [id=83ad467bec3d46f8b879b7caf6ba6319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:41.971319 32122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e2f618ab1b4180aa6e70b888b9c5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:41.974287 32061 tablet_bootstrap.cc:492] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Bootstrap starting.
I20260807 18:41:41.975275 32061 tablet_bootstrap.cc:654] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:41.975365 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.969860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:40983 (local address 127.0.0.1:52580)
0807 18:41:41.970133 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:41.970148 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:41.970246 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:41.971302 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:41.971306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:41.971327 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:41.971413 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.971419 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.972949 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.972952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.973739 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.973748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.974868 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.974871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.974998 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.975054 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 18:41:41.975110 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 18:41:41.975194 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":130,"threads_started":1}
I20260807 18:41:41.975364 32234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.969416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:46749 (local address 127.0.0.1:52938)
0807 18:41:41.969561 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:41.969576 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:41.969686 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:41.970852 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:41.970855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:41.970875 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:41.970982 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.970988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.972725 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.972728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.973955 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:41.973969 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.975016 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.975019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:41.975123 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.975141 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:41:41.975182 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 18:41:41.975242 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:41:41.976220 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.970355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52580 (local address 127.6.138.67:40983)
0807 18:41:41.970795 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:41.970810 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:41.970835 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:41:41.970909 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:41.970915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:41.971137 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:41.971562 (+ 425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:41.971746 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.972830 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.973866 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.974714 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.975816 (+ 1102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.975942 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:41.975986 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 18:41:41.976068 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":186,"threads_started":1}
I20260807 18:41:41.976220 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:41.969886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52938 (local address 127.6.138.66:46749)
0807 18:41:41.970318 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:41.970331 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:41.970357 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:41:41.970437 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:41.970441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:41.970667 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:41.971147 (+ 480us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:41.971344 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.972596 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.974113 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:41.974892 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:41.975733 (+ 841us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:41.975902 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:41.975964 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:41:41.976061 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":166,"threads_started":1}
I20260807 18:41:41.976962 32061 tablet_bootstrap.cc:492] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: No bootstrap required, opened a new log
I20260807 18:41:41.977089 32061 ts_tablet_manager.cc:1397] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:41:41.977560 32061 raft_consensus.cc:348] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:41:41.977691 32061 raft_consensus.cc:374] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:41.977732 32061 raft_consensus.cc:729] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ad19809f887452bb6a409877ee471c6, State: Initialized, Role: FOLLOWER
I20260807 18:41:41.979707 32331 tablet_service.cc:1467] Processing CreateTablet for tablet 05e2f618ab1b4180aa6e70b888b9c5be (DEFAULT_TABLE table=test-table [id=83ad467bec3d46f8b879b7caf6ba6319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:41.979682 32551 tablet_service.cc:1467] Processing CreateTablet for tablet 05e2f618ab1b4180aa6e70b888b9c5be (DEFAULT_TABLE table=test-table [id=83ad467bec3d46f8b879b7caf6ba6319]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:41:41.980468 32061 consensus_queue.cc:260] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:41:41.980780 32061 ts_tablet_manager.cc:1428] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260807 18:41:41.980906 32331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e2f618ab1b4180aa6e70b888b9c5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:41.980926 32551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e2f618ab1b4180aa6e70b888b9c5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:41:41.989790 32689 tablet_bootstrap.cc:492] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: Bootstrap starting.
I20260807 18:41:41.990572 32691 tablet_bootstrap.cc:492] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: Bootstrap starting.
I20260807 18:41:41.992913 32689 tablet_bootstrap.cc:654] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:41.993310 32691 tablet_bootstrap.cc:654] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: Neither blocks nor log segments found. Creating new log.
I20260807 18:41:41.993945 32689 log.cc:824] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:41.994077 32691 log.cc:824] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: Log is configured to *not* fsync() on all Append() calls
I20260807 18:41:41.995939 32691 tablet_bootstrap.cc:492] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: No bootstrap required, opened a new log
I20260807 18:41:41.995942 32689 tablet_bootstrap.cc:492] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: No bootstrap required, opened a new log
I20260807 18:41:41.996150 32689 ts_tablet_manager.cc:1397] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 18:41:41.996160 32691 ts_tablet_manager.cc:1397] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:41:41.999155 32689 raft_consensus.cc:348] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:41:41.999217 32691 raft_consensus.cc:348] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:41:41.999475 32689 raft_consensus.cc:374] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:41.999505 32691 raft_consensus.cc:374] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:41:41.999536 32689 raft_consensus.cc:729] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e52ea3713d747e4bc681e4e227097bd, State: Initialized, Role: FOLLOWER
I20260807 18:41:41.999567 32691 raft_consensus.cc:729] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7570f9b221446cd875c4b0b208d3389, State: Initialized, Role: FOLLOWER
I20260807 18:41:42.000025 32691 consensus_queue.cc:260] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:41:42.000015 32689 consensus_queue.cc:260] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:41:42.000768 32441 heartbeater.cc:503] Master 127.6.138.126:38847 was elected leader, sending a full tablet report...
I20260807 18:41:42.000775 32665 heartbeater.cc:503] Master 127.6.138.126:38847 was elected leader, sending a full tablet report...
I20260807 18:41:42.001173 32691 ts_tablet_manager.cc:1428] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 18:41:42.001173 32689 ts_tablet_manager.cc:1428] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 18:41:42.027570 32699 raft_consensus.cc:482] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:41:42.027870 32699 raft_consensus.cc:504] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:41:42.029237 32699 leader_election.cc:288] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983), 7ad19809f887452bb6a409877ee471c6 (127.6.138.65:34031)
I20260807 18:41:42.033815 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:42.029680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:38841 (local address 127.6.138.65:34031)
0807 18:41:42.029998 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:42.030002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:42.030022 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:41:42.030072 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:42.030074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:42.030130 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:42.030255 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:42.030557 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.031318 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.032790 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.033251 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.033390 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:42.033565 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:42.033618 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 18:41:42.033687 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":163,"threads_started":1}
I20260807 18:41:42.033813 32443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:42.029501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34031 (local address 127.6.138.66:38841)
0807 18:41:42.029671 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:42.029690 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:42.029798 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:42.030284 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:42.030288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:42.030307 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:42.030431 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:42.030437 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.031443 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.031447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:42.032652 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:42.032663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.033379 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.033382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:42.033500 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:42.033521 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:41:42.033587 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 18:41:42.033665 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":63}
I20260807 18:41:42.033804 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:42.029617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:55751 (local address 127.6.138.67:40983)
0807 18:41:42.029766 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:42.029771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:42.030199 (+ 428us) server_negotiation.cc:407] Connection header received
0807 18:41:42.030349 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:42.030352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:42.030402 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:42.030493 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:41:42.030771 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.031315 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.032793 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.033210 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.033349 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:42.033408 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:42.033597 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 18:41:42.033673 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:41:42.033820 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:42.029502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:40983 (local address 127.6.138.66:55751)
0807 18:41:42.030098 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:42.030109 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:41:42.030231 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:41:42.030515 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:41:42.030518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:41:42.030534 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:41:42.030657 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:42.030662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.031505 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.031508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:42.032662 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:41:42.032672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.033345 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.033348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:41:42.033449 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:42.033485 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:41:42.033577 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:41:42.033657 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":164,"thread_start_us":160,"threads_started":1}
I20260807 18:41:42.034636 32586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2f618ab1b4180aa6e70b888b9c5be" candidate_uuid: "3e52ea3713d747e4bc681e4e227097bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7570f9b221446cd875c4b0b208d3389" is_pre_election: true
I20260807 18:41:42.034735 32152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2f618ab1b4180aa6e70b888b9c5be" candidate_uuid: "3e52ea3713d747e4bc681e4e227097bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ad19809f887452bb6a409877ee471c6" is_pre_election: true
I20260807 18:41:42.035013 32586 raft_consensus.cc:2454] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e52ea3713d747e4bc681e4e227097bd in term 0.
I20260807 18:41:42.035059 32152 raft_consensus.cc:2454] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e52ea3713d747e4bc681e4e227097bd in term 0.
I20260807 18:41:42.035557 32261 leader_election.cc:302] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e52ea3713d747e4bc681e4e227097bd, 7ad19809f887452bb6a409877ee471c6; no voters:
I20260807 18:41:42.036059 32699 raft_consensus.cc:2781] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:41:42.036154 32699 raft_consensus.cc:482] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:41:42.036187 32699 raft_consensus.cc:3037] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:42.037463 32699 raft_consensus.cc:504] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:41:42.037827 32699 leader_election.cc:288] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [CANDIDATE]: Term 1 election: Requested vote from peers f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983), 7ad19809f887452bb6a409877ee471c6 (127.6.138.65:34031)
I20260807 18:41:42.038364 32586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2f618ab1b4180aa6e70b888b9c5be" candidate_uuid: "3e52ea3713d747e4bc681e4e227097bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7570f9b221446cd875c4b0b208d3389"
I20260807 18:41:42.038394 32152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2f618ab1b4180aa6e70b888b9c5be" candidate_uuid: "3e52ea3713d747e4bc681e4e227097bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ad19809f887452bb6a409877ee471c6"
I20260807 18:41:42.038525 32586 raft_consensus.cc:3037] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:42.038558 32152 raft_consensus.cc:3037] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:41:42.039665 32586 raft_consensus.cc:2454] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e52ea3713d747e4bc681e4e227097bd in term 1.
I20260807 18:41:42.039693 32152 raft_consensus.cc:2454] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e52ea3713d747e4bc681e4e227097bd in term 1.
I20260807 18:41:42.040067 32261 leader_election.cc:302] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e52ea3713d747e4bc681e4e227097bd, 7ad19809f887452bb6a409877ee471c6; no voters:
I20260807 18:41:42.040292 32699 raft_consensus.cc:2781] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:41:42.040577 32699 raft_consensus.cc:686] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 1 LEADER]: Becoming Leader. State: Replica: 3e52ea3713d747e4bc681e4e227097bd, State: Running, Role: LEADER
I20260807 18:41:42.040956 32699 consensus_queue.cc:237] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:41:42.044618 31927 catalog_manager.cc:5495] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66). New cstate: current_term: 1 leader_uuid: "3e52ea3713d747e4bc681e4e227097bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } health_report { overall_health: UNKNOWN } } }
I20260807 18:41:42.056363 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:42.052588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.6.138.66:46749)
0807 18:41:42.052727 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:41:42.052732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:41:42.052829 (+ 97us) server_negotiation.cc:407] Connection header received
0807 18:41:42.053030 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:41:42.053033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:41:42.053098 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:41:42.053218 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:41:42.053462 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.053964 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.054817 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:41:42.055033 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:41:42.055152 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:41:42.055507 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:41:42.055912 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:41:42.056048 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:41:42.056122 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 18:41:42.056179 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:41:42.062657 32152 raft_consensus.cc:1261] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Refusing update from remote peer 3e52ea3713d747e4bc681e4e227097bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:41:42.062677 32586 raft_consensus.cc:1261] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [term 1 FOLLOWER]: Refusing update from remote peer 3e52ea3713d747e4bc681e4e227097bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:41:42.063473 32704 consensus_queue.cc:1035] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:41:42.063673 32699 consensus_queue.cc:1035] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:41:42.069051 32708 mvcc.cc:205] Tried to move back new op lower bound from 7315980706040909824 to 7315980705962762240. Current Snapshot: MvccSnapshot[applied={T|T < 7315980706040909824}]
I20260807 18:41:42.070881 32709 mvcc.cc:205] Tried to move back new op lower bound from 7315980706040909824 to 7315980705962762240. Current Snapshot: MvccSnapshot[applied={T|T < 7315980706040909824}]
I20260807 18:41:42.074172 32710 mvcc.cc:205] Tried to move back new op lower bound from 7315980706040909824 to 7315980705962762240. Current Snapshot: MvccSnapshot[applied={T|T < 7315980706040909824}]
I20260807 18:41:42.078814 31927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:41:42.079042 31927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:41:42.081575 31927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Sending DeleteTablet for 3 replicas of tablet 05e2f618ab1b4180aa6e70b888b9c5be
I20260807 18:41:42.082441 32551 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e2f618ab1b4180aa6e70b888b9c5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:42 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:52580
I20260807 18:41:42.082494 32122 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e2f618ab1b4180aa6e70b888b9c5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:42 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:45006
I20260807 18:41:42.082592 32331 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e2f618ab1b4180aa6e70b888b9c5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:41:42 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:52938
I20260807 18:41:42.083039 32725 tablet_replica.cc:330] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: stopping tablet replica
I20260807 18:41:42.083086 32726 tablet_replica.cc:330] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: stopping tablet replica
I20260807 18:41:42.083117 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 31995
I20260807 18:41:42.083421 32725 raft_consensus.cc:2229] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:41:42.086023 32725 raft_consensus.cc:2258] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:42.103252 32726 raft_consensus.cc:2229] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:41:42.103888 32726 raft_consensus.cc:2258] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:41:42.106640 32725 ts_tablet_manager.cc:1905] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:42.107093 32726 ts_tablet_manager.cc:1905] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:41:42.110105 32726 ts_tablet_manager.cc:1918] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:42.110239 32726 log.cc:1196] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/05e2f618ab1b4180aa6e70b888b9c5be
I20260807 18:41:42.110301 32725 ts_tablet_manager.cc:1918] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:41:42.110435 32725 log.cc:1196] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/05e2f618ab1b4180aa6e70b888b9c5be
I20260807 18:41:42.110797 32726 ts_tablet_manager.cc:1939] T 05e2f618ab1b4180aa6e70b888b9c5be P 3e52ea3713d747e4bc681e4e227097bd: Deleting consensus metadata
I20260807 18:41:42.111028 32725 ts_tablet_manager.cc:1939] T 05e2f618ab1b4180aa6e70b888b9c5be P f7570f9b221446cd875c4b0b208d3389: Deleting consensus metadata
I20260807 18:41:42.112344 31911 catalog_manager.cc:4841] TS f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983): tablet 05e2f618ab1b4180aa6e70b888b9c5be (table test-table [id=83ad467bec3d46f8b879b7caf6ba6319]) successfully deleted
I20260807 18:41:42.112344 31909 catalog_manager.cc:4841] TS 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749): tablet 05e2f618ab1b4180aa6e70b888b9c5be (table test-table [id=83ad467bec3d46f8b879b7caf6ba6319]) successfully deleted
W20260807 18:41:42.121814 31909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:34031 (error 108)
I20260807 18:41:42.122486 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 32231
I20260807 18:41:42.125977 32234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:41:42.125570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34031 (local address 127.0.0.1:45010)
0807 18:41:42.125769 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 18:41:42.125861 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 18:41:42.126905 31909 catalog_manager.cc:4584] TS 7ad19809f887452bb6a409877ee471c6 (127.6.138.65:34031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05e2f618ab1b4180aa6e70b888b9c5be: Network error: Client connection negotiation failed: client connection to 127.6.138.65:34031: connect: Connection refused (error 111)
I20260807 18:41:42.135612 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 32445
I20260807 18:41:42.155150 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 31883
I20260807 18:42:02.170625 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38847
--webserver_interface=127.6.138.126
--webserver_port=37135
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:38847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:02.290921 32730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:02.291213 32730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:02.291256 32730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:02.295840 32730 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:42:02.295915 32730 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:42:02.295933 32730 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:42:02.295979 32730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:02.296017 32730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:02.296087 32730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:02.296123 32730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:02.296151 32730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:02.303009 32730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:38847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=37135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:02.304236 32730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:02.305449 32730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:02.314981 32737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:02.315099 32730 server_base.cc:1034] running on GCE node
W20260807 18:42:02.315003 32738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:02.315006 32740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:02.315944 32730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:02.317282 32730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:02.318454 32730 hybrid_clock.cc:648] HybridClock initialized: now 1786128122318434 us; error 34 us; skew 500 ppm
Aug 07 18:42:02 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128122, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:02.329166 32730 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:02.330241 32730 webserver.cc:460] Webserver started at http://127.6.138.126:37135/ using document root <none> and password file <none>
I20260807 18:42:02.330821 32730 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:02.330891 32730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:02.334983 32730 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:42:02.338760 32750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:02.340430 32730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:42:02.340592 32730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "be75824486274c2e8358a42d753ec1de"
format_stamp: "Formatted at 2026-08-07 18:41:19 on dist-test-slave-2d19"
server_key: "b8682c3c67bff05ca7043e5e9926b665"
server_key_iv: "0086aa29ab24c9ce1a76b109b57f9017"
server_key_version: "encryptionkey@0"
I20260807 18:42:02.341287 32730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:02.361127 32730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:02.364948 32730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:02.365180 32730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:02.373028 32730 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:38847
I20260807 18:42:02.373347 370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:38847 every 8 connection(s)
I20260807 18:42:02.374199 32730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:02.378911 371 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:42:02.379465 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 32730
I20260807 18:42:02.379930 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:34031
--local_ip_for_outbound_sockets=127.6.138.65
--tserver_master_addrs=127.6.138.126:38847
--webserver_port=40769
--webserver_interface=127.6.138.65
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:42:02.381014 371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Bootstrap starting.
I20260807 18:42:02.401410 371 log.cc:824] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:02.413966 371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:42:02.414400 371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Bootstrap complete.
I20260807 18:42:02.419116 371 raft_consensus.cc:348] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:42:02.419535 371 raft_consensus.cc:729] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: be75824486274c2e8358a42d753ec1de, State: Initialized, Role: FOLLOWER
I20260807 18:42:02.420087 371 consensus_queue.cc:260] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:42:02.420215 371 raft_consensus.cc:388] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:02.420339 371 raft_consensus.cc:482] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:02.420467 371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:42:02.422538 371 raft_consensus.cc:504] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:42:02.422894 371 leader_election.cc:302] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be75824486274c2e8358a42d753ec1de; no voters:
I20260807 18:42:02.423239 371 leader_election.cc:288] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:42:02.423382 379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:42:02.423640 379 raft_consensus.cc:686] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [term 3 LEADER]: Becoming Leader. State: Replica: be75824486274c2e8358a42d753ec1de, State: Running, Role: LEADER
I20260807 18:42:02.423940 379 consensus_queue.cc:237] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } }
I20260807 18:42:02.424660 371 sys_catalog.cc:564] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:02.425585 381 sys_catalog.cc:455] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "be75824486274c2e8358a42d753ec1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } } }
I20260807 18:42:02.425750 381 sys_catalog.cc:458] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:02.426388 383 sys_catalog.cc:455] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: SysCatalogTable state changed. Reason: New leader be75824486274c2e8358a42d753ec1de. Latest consensus state: current_term: 3 leader_uuid: "be75824486274c2e8358a42d753ec1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be75824486274c2e8358a42d753ec1de" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38847 } } }
I20260807 18:42:02.426601 383 sys_catalog.cc:458] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:02.426990 392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:02.431536 392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:02.432139 392 catalog_manager.cc:1259] Loaded cluster ID: d84cf33a9ef64897a8ddd37bf17b5f1d
I20260807 18:42:02.432226 392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:02.434222 392 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:02.434904 392 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Loaded TSK: 0
I20260807 18:42:02.435005 392 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be75824486274c2e8358a42d753ec1de: Loaded TSK: 1
F20260807 18:42:02.435743 392 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f64905949ad google::LogMessage::Fail() at ??:0
@ 0x7f649059788a google::LogMessage::SendToLog() at ??:0
@ 0x7f64905944de google::LogMessage::Flush() at ??:0
@ 0x7f6490595a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f649325818d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6493250f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f649aef97bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f649aed399c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f649aed4178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f649af15df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f649af3482b std::function<>::operator()() at ??:0
@ 0x7f649aed3b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f649aed4f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f649aed1579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f649af15640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f64927e63d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f64927e6c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f64927e83f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f64927d7512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6493e1b6db start_thread at ??:0
@ 0x7f648f09261f clone at ??:0
W20260807 18:42:02.532721 376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:02.533013 376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:02.533061 376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:02.537429 376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:02.537521 376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:02.537632 376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:42:02.543267 376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:34031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=40769
--tserver_master_addrs=127.6.138.126:38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:02.544504 376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:02.545610 376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:02.555872 447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:02.556046 444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:02.555872 442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:02.556167 376 server_base.cc:1034] running on GCE node
I20260807 18:42:02.556771 376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:02.557483 376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:02.558667 376 hybrid_clock.cc:648] HybridClock initialized: now 1786128122558645 us; error 35 us; skew 500 ppm
Aug 07 18:42:02 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128122, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:02.561573 376 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:42:02.562695 376 webserver.cc:460] Webserver started at http://127.6.138.65:40769/ using document root <none> and password file <none>
I20260807 18:42:02.563366 376 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:02.563436 376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:02.567544 376 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:42:02.569919 455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:02.571242 376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:42:02.571341 376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "7ad19809f887452bb6a409877ee471c6"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "97d1513c76c343f72ee184454626fe5e"
server_key_iv: "04132699bbca835323d03b74a495699e"
server_key_version: "encryptionkey@0"
I20260807 18:42:02.571949 376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:02.592505 376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:02.596065 376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:02.596314 376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:02.596982 376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:02.598199 466 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 18:42:02.604051 376 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:42:02.604153 376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 18:42:02.604242 376 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:42:02.606092 376 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:42:02.606170 376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 18:42:02.606532 466 tablet_bootstrap.cc:492] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Bootstrap starting.
I20260807 18:42:02.617810 376 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:34031
I20260807 18:42:02.617888 625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:34031 every 8 connection(s)
I20260807 18:42:02.618957 376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:42:02.629036 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 376
I20260807 18:42:02.629537 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:46749
--local_ip_for_outbound_sockets=127.6.138.66
--tserver_master_addrs=127.6.138.126:38847
--webserver_port=34857
--webserver_interface=127.6.138.66
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:42:02.630625 466 log.cc:824] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:02.638257 466 tablet_bootstrap.cc:492] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:42:02.638664 466 tablet_bootstrap.cc:492] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Bootstrap complete.
I20260807 18:42:02.639878 466 ts_tablet_manager.cc:1397] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.004s
I20260807 18:42:02.642560 466 raft_consensus.cc:348] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:42:02.643014 466 raft_consensus.cc:729] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ad19809f887452bb6a409877ee471c6, State: Initialized, Role: FOLLOWER
I20260807 18:42:02.643532 466 consensus_queue.cc:260] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:42:02.644601 466 ts_tablet_manager.cc:1428] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260807 18:42:02.714022 630 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:42:02.621057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:36005)
0807 18:42:02.621570 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:02.621627 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:02.622483 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:02.713187 (+ 90704us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":139,"threads_started":1}
W20260807 18:42:02.714622 626 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:38847 (0 consecutive failures): Network error: Failed to ping master at 127.6.138.126:38847: Client connection negotiation failed: client connection to 127.6.138.126:38847: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:42:02.715422 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:02.715163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:53747)
0807 18:42:02.715301 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:02.715365 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:42:02.716091 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:02.715889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:36163)
0807 18:42:02.715992 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:02.716045 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 18:42:02.716261 626 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:42:02.772583 632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:02.772876 632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:02.772950 632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:02.777129 632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:02.777215 632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:02.777304 632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:42:02.782737 632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:46749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=34857
--tserver_master_addrs=127.6.138.126:38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:02.783872 632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:02.784962 632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:02.795035 641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:02.795060 642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:02.795133 644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:02.795459 632 server_base.cc:1034] running on GCE node
I20260807 18:42:02.795928 632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:02.796674 632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:02.797878 632 hybrid_clock.cc:648] HybridClock initialized: now 1786128122797857 us; error 31 us; skew 500 ppm
Aug 07 18:42:02 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128122, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:02.800872 632 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:42:02.801827 632 webserver.cc:460] Webserver started at http://127.6.138.66:34857/ using document root <none> and password file <none>
I20260807 18:42:02.802428 632 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:02.802527 632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:02.806593 632 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:42:02.808804 657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:02.810053 632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:02.810191 632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "3e52ea3713d747e4bc681e4e227097bd"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "fd95cae78186bdd38bb6e1c3a1b3c3bc"
server_key_iv: "bd0fb099e65365efa20e31a1ddb0596c"
server_key_version: "encryptionkey@0"
I20260807 18:42:02.810755 632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:02.831640 632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:02.835309 632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:02.835547 632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:02.836216 632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:02.837165 632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:02.837241 632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:02.837319 632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:02.837365 632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:02.847268 632 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:46749
I20260807 18:42:02.847289 834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:46749 every 8 connection(s)
I20260807 18:42:02.848464 632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:42:02.849579 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 632
I20260807 18:42:02.850086 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:40983
--local_ip_for_outbound_sockets=127.6.138.67
--tserver_master_addrs=127.6.138.126:38847
--webserver_port=44249
--webserver_interface=127.6.138.67
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:37967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:42:02.853368 838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:02.851150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:57569)
0807 18:42:02.851742 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:02.851928 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":131,"threads_started":1}
W20260807 18:42:02.853847 836 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:38847 (0 consecutive failures): Network error: Failed to ping master at 127.6.138.126:38847: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
I20260807 18:42:02.854640 838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:02.854379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:34231)
0807 18:42:02.854529 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:02.854583 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:42:02.855424 838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:02.855173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:52933)
0807 18:42:02.855304 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:02.855357 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:42:02.855618 836 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:42:02.969220 839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:02.969511 839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:02.969583 839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:02.973595 839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:02.973685 839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:02.973768 839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:42:02.979285 839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:37967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:40983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=44249
--tserver_master_addrs=127.6.138.126:38847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:02.980485 839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:02.981575 839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:02.991420 848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:02.991447 847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:02.991639 839 server_base.cc:1034] running on GCE node
W20260807 18:42:02.991722 850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:02.992131 839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:02.992853 839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:02.994060 839 hybrid_clock.cc:648] HybridClock initialized: now 1786128122994046 us; error 26 us; skew 500 ppm
Aug 07 18:42:02 dist-test-slave-2d19 krb5kdc[31044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128122, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:02.996963 839 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:42:02.997888 839 webserver.cc:460] Webserver started at http://127.6.138.67:44249/ using document root <none> and password file <none>
I20260807 18:42:02.998499 839 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:02.998596 839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:03.002664 839 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:42:03.005030 859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:03.006294 839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:42:03.006443 839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "f7570f9b221446cd875c4b0b208d3389"
format_stamp: "Formatted at 2026-08-07 18:41:20 on dist-test-slave-2d19"
server_key: "55d019242d01b87a2b9e18dbcd790386"
server_key_iv: "8c3a647756383ea41d2fdb3dd9942610"
server_key_version: "encryptionkey@0"
I20260807 18:42:03.007082 839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:03.021714 839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:03.024951 839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:03.025172 839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:03.025794 839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:03.026649 839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:03.026723 839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:03.026796 839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:03.026840 839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:03.037521 839 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:40983
I20260807 18:42:03.037544 1028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:40983 every 8 connection(s)
I20260807 18:42:03.038614 839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:42:03.039167 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 839
I20260807 18:42:03.042924 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:03.041026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:38289)
0807 18:42:03.041575 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:03.041682 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":140,"threads_started":1}
W20260807 18:42:03.043414 1031 heartbeater.cc:644] Failed to heartbeat to 127.6.138.126:38847 (0 consecutive failures): Network error: Failed to ping master at 127.6.138.126:38847: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
I20260807 18:42:03.044086 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:03.043798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:43887)
0807 18:42:03.043975 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:03.044028 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:42:03.044876 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:03.044628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:33425)
0807 18:42:03.044754 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:03.044815 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 18:42:03.045136 1031 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:42:03.717346 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:03.716886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:33495)
0807 18:42:03.717160 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:03.717225 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":86,"threads_started":1}
I20260807 18:42:03.856628 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:03.856180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:36067)
0807 18:42:03.856471 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:03.856526 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":128,"threads_started":1}
I20260807 18:42:04.011474 1039 raft_consensus.cc:482] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:04.011710 1039 raft_consensus.cc:504] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:42:04.012852 1039 leader_election.cc:288] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749), f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983)
Aug 07 18:42:04 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128122, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.66@KRBTEST.COM
Aug 07 18:42:04 dist-test-slave-2d19 krb5kdc[31044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128122, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.67@KRBTEST.COM
I20260807 18:42:04.028977 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:04.013101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:40983 (local address 127.6.138.65:57075)
0807 18:42:04.013487 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:04.013502 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:04.013649 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:04.016024 (+ 2375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:04.016026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:04.016192 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:04.016619 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:04.016636 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:04.017923 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:04.017932 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:04.019061 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:04.019071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:04.019773 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:04.019776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:04.020008 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:04.020694 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:04.020711 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:04.022752 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:04.025515 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:04.025522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:04.025538 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:04.026133 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:04.026599 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:04.026603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:04.026605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:04.026823 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:04.027664 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:04.027672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:04.028058 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 18:42:04.028336 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 18:42:04.028809 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":296,"spinlock_wait_cycles":19712,"thread_start_us":80,"threads_started":1}
I20260807 18:42:04.029317 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:04.013273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:36983 (local address 127.6.138.66:46749)
0807 18:42:04.013726 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:04.013747 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:04.013782 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:42:04.014715 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:04.014722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:04.015191 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:04.015532 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:04.016810 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:04.017771 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:04.019197 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:04.019661 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:04.019839 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:04.022664 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:04.022668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:04.022681 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:04.022709 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:04.025333 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:04.026296 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:04.026303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:04.026309 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:04.026501 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:04.026940 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:04.026944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:04.026945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:04.027530 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:04.027671 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:04.028519 (+ 848us) server_negotiation.cc:299] Negotiation successful
0807 18:42:04.028837 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":100,"threads_started":1}
I20260807 18:42:04.029034 1042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:04.013275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:57075 (local address 127.6.138.67:40983)
0807 18:42:04.013685 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:04.013701 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:04.013749 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:42:04.014673 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:04.014683 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:04.015192 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:04.015532 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:04.016803 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:04.017776 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:04.019189 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:04.019646 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:04.019821 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:04.022970 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:04.022985 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:04.023004 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:04.023046 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:04.025378 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:04.026297 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:04.026303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:04.026309 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:04.026471 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:04.026992 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:04.026999 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:04.027002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:04.027524 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:04.027671 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:04.028512 (+ 841us) server_negotiation.cc:299] Negotiation successful
0807 18:42:04.028837 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":104,"threads_started":1}
I20260807 18:42:04.029016 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:04.013074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:46749 (local address 127.6.138.65:36983)
0807 18:42:04.013284 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:04.013299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:04.013415 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:04.015613 (+ 2198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:04.015618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:04.016008 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:04.016619 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:04.016636 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:04.017935 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:04.017939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:04.019062 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:04.019073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:04.019790 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:04.019794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:04.019991 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:04.020756 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:04.020768 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:04.022424 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:04.025515 (+ 3091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:04.025522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:04.025538 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:04.026133 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:04.026599 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:04.026603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:04.026605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:04.026823 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:04.027664 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:04.027672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:04.028058 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 18:42:04.028336 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 18:42:04.028754 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":221,"spinlock_wait_cycles":55296}
W20260807 18:42:04.032163 459 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
W20260807 18:42:04.032362 456 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
I20260807 18:42:04.032485 456 leader_election.cc:302] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ad19809f887452bb6a409877ee471c6; no voters: 3e52ea3713d747e4bc681e4e227097bd, f7570f9b221446cd875c4b0b208d3389
I20260807 18:42:04.032804 1039 raft_consensus.cc:2726] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:42:04.046175 1044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:04.045657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:45163)
0807 18:42:04.046005 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:04.046069 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":85,"threads_started":1}
I20260807 18:42:04.718688 1045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:04.718251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:49665)
0807 18:42:04.718526 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:04.718589 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":81,"threads_started":1}
I20260807 18:42:04.857798 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:04.857364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:34337)
0807 18:42:04.857646 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:04.857703 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
I20260807 18:42:05.047462 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:05.047006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:59279)
0807 18:42:05.047279 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:05.047337 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":81,"threads_started":1}
I20260807 18:42:05.719846 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:05.719445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:59557)
0807 18:42:05.719692 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:05.719749 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":80,"threads_started":1}
I20260807 18:42:05.859172 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:05.858666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:42947)
0807 18:42:05.859007 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:05.859072 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":84,"threads_started":1}
I20260807 18:42:05.892143 1055 raft_consensus.cc:482] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:05.892366 1055 raft_consensus.cc:504] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:42:05.892825 1055 leader_election.cc:288] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749), f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983)
W20260807 18:42:05.893800 459 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
W20260807 18:42:05.893945 456 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
I20260807 18:42:05.894039 456 leader_election.cc:302] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ad19809f887452bb6a409877ee471c6; no voters: 3e52ea3713d747e4bc681e4e227097bd, f7570f9b221446cd875c4b0b208d3389
I20260807 18:42:05.894212 1055 raft_consensus.cc:2726] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:42:06.049049 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:06.048387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:59009)
0807 18:42:06.048783 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:06.048936 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":102,"threads_started":1}
I20260807 18:42:06.721098 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:06.720651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:40801)
0807 18:42:06.720935 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:06.720997 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":77,"threads_started":1}
I20260807 18:42:06.860564 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:06.860035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:49265)
0807 18:42:06.860382 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:06.860443 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":83,"threads_started":1}
I20260807 18:42:07.050316 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:07.049836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:43915)
0807 18:42:07.050139 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:07.050196 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":93,"threads_started":1}
I20260807 18:42:07.722532 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:07.722072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:48071)
0807 18:42:07.722357 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:07.722415 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":77,"threads_started":1}
I20260807 18:42:07.861929 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:07.861448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:44595)
0807 18:42:07.861710 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:07.861768 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260807 18:42:08.051754 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:08.051280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:45429)
0807 18:42:08.051594 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:08.051654 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":85,"threads_started":1}
I20260807 18:42:08.723784 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:08.723344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:46465)
0807 18:42:08.723624 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:08.723683 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
I20260807 18:42:08.863592 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:08.862932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:51665)
0807 18:42:08.863323 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:08.863416 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":147,"threads_started":1}
I20260807 18:42:08.900168 1067 raft_consensus.cc:482] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:08.900388 1067 raft_consensus.cc:504] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:42:08.900949 1067 leader_election.cc:288] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749), f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983)
W20260807 18:42:08.901914 456 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
W20260807 18:42:08.902005 459 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
I20260807 18:42:08.902053 456 leader_election.cc:302] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ad19809f887452bb6a409877ee471c6; no voters: 3e52ea3713d747e4bc681e4e227097bd, f7570f9b221446cd875c4b0b208d3389
I20260807 18:42:08.902217 1067 raft_consensus.cc:2726] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:42:09.053171 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:09.052690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:41355)
0807 18:42:09.052998 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:09.053065 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":98,"threads_started":1}
I20260807 18:42:09.725126 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:09.724646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:53901)
0807 18:42:09.724967 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:09.725025 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":83,"threads_started":1}
I20260807 18:42:09.865137 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:09.864542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:49817)
0807 18:42:09.864956 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:09.865034 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":86,"threads_started":1}
I20260807 18:42:10.054515 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:10.054029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:38151)
0807 18:42:10.054357 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:10.054413 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":101,"threads_started":1}
I20260807 18:42:10.726490 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:10.726028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:39125)
0807 18:42:10.726301 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:10.726359 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260807 18:42:10.866622 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:10.866117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:59417)
0807 18:42:10.866415 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:10.866520 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":83,"threads_started":1}
I20260807 18:42:10.875592 1079 raft_consensus.cc:482] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:10.875730 1079 raft_consensus.cc:504] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:42:10.876355 1079 leader_election.cc:288] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749), f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983)
W20260807 18:42:10.877220 456 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
W20260807 18:42:10.877331 459 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
I20260807 18:42:10.877435 459 leader_election.cc:302] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ad19809f887452bb6a409877ee471c6; no voters: 3e52ea3713d747e4bc681e4e227097bd, f7570f9b221446cd875c4b0b208d3389
I20260807 18:42:10.877602 1079 raft_consensus.cc:2726] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:42:11.055825 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:11.055394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:33295)
0807 18:42:11.055671 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:11.055727 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":87,"threads_started":1}
I20260807 18:42:11.727743 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:11.727331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:42291)
0807 18:42:11.727586 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:11.727643 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":83,"threads_started":1}
I20260807 18:42:11.867870 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:11.867441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:57753)
0807 18:42:11.867714 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:11.867769 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":96,"threads_started":1}
I20260807 18:42:12.057199 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:12.056734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:55951)
0807 18:42:12.057036 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:12.057098 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":82,"threads_started":1}
I20260807 18:42:12.729230 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:12.728769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:59241)
0807 18:42:12.729069 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:12.729129 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":105,"threads_started":1}
I20260807 18:42:12.869150 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:12.868688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:38217)
0807 18:42:12.868986 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:12.869047 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":82,"threads_started":1}
I20260807 18:42:13.058550 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:13.058075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:40137)
0807 18:42:13.058384 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:13.058446 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":109,"threads_started":1}
I20260807 18:42:13.343087 1089 raft_consensus.cc:482] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:13.343271 1089 raft_consensus.cc:504] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e52ea3713d747e4bc681e4e227097bd" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 46749 } } peers { permanent_uuid: "f7570f9b221446cd875c4b0b208d3389" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 40983 } } peers { permanent_uuid: "7ad19809f887452bb6a409877ee471c6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34031 } }
I20260807 18:42:13.343765 1089 leader_election.cc:288] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749), f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983)
W20260807 18:42:13.344657 456 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3e52ea3713d747e4bc681e4e227097bd (127.6.138.66:46749): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
W20260807 18:42:13.344774 459 leader_election.cc:341] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7570f9b221446cd875c4b0b208d3389 (127.6.138.67:40983): Not found: Tablet not found: 05e2f618ab1b4180aa6e70b888b9c5be
I20260807 18:42:13.344895 459 leader_election.cc:302] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ad19809f887452bb6a409877ee471c6; no voters: 3e52ea3713d747e4bc681e4e227097bd, f7570f9b221446cd875c4b0b208d3389
I20260807 18:42:13.345057 1089 raft_consensus.cc:2726] T 05e2f618ab1b4180aa6e70b888b9c5be P 7ad19809f887452bb6a409877ee471c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:42:13.730597 1090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:13.730138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:58195)
0807 18:42:13.730429 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:13.730494 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":82,"threads_started":1}
I20260807 18:42:13.870460 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:13.869994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:43369)
0807 18:42:13.870273 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:13.870331 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":88,"threads_started":1}
I20260807 18:42:14.059871 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:14.059428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:49879)
0807 18:42:14.059717 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:14.059774 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
I20260807 18:42:14.731801 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:14.731384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:57991)
0807 18:42:14.731652 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:14.731704 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
I20260807 18:42:14.871860 1095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:14.871400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:53945)
0807 18:42:14.871685 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:14.871748 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":89,"threads_started":1}
I20260807 18:42:15.061401 1096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:15.060821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:43521)
0807 18:42:15.061161 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:15.061268 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":128,"threads_started":1}
I20260807 18:42:15.733237 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:15.732741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:36975)
0807 18:42:15.733075 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:15.733137 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":104,"threads_started":1}
I20260807 18:42:15.873211 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:15.872733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:35409)
0807 18:42:15.873046 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:15.873110 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260807 18:42:16.062847 1101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:16.062381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:53363)
0807 18:42:16.062685 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:16.062747 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":117,"threads_started":1}
I20260807 18:42:16.734615 1103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:16.734162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:42267)
0807 18:42:16.734445 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:16.734513 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":86,"threads_started":1}
I20260807 18:42:16.874637 1105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:16.874159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:45559)
0807 18:42:16.874461 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:16.874520 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":103,"threads_started":1}
I20260807 18:42:17.064177 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:17.063666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:42231)
0807 18:42:17.064002 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:17.064070 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":83,"threads_started":1}
I20260807 18:42:17.736405 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:17.735827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.65:56435)
0807 18:42:17.736201 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:17.736297 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260807 18:42:17.876396 1108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:17.875874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.66:39225)
0807 18:42:17.876206 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:17.876296 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":116,"threads_started":1}
I20260807 18:42:18.043831 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 376
I20260807 18:42:18.056190 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 632
I20260807 18:42:18.065739 1111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:18.065300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:38847 (local address 127.6.138.67:60719)
0807 18:42:18.065570 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:18.065642 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.126:38847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":96,"threads_started":1}
I20260807 18:42:18.070448 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 839
2026-08-07T18:42:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59806 ms)
[----------] 30 tests from SecurityITest (256757 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:18 dist-test-slave-2d19 krb5kdc[1116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:18 dist-test-slave-2d19 krb5kdc[1116](info): set up 2 sockets
Aug 07 18:42:18 dist-test-slave-2d19 krb5kdc[1116](info): commencing operation
krb5kdc: starting...
W20260807 18:42:19.341763 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:19 dist-test-slave-2d19 krb5kdc[1116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:42:19.480145 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:39613
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:39613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:19.598217 1133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:19.598482 1133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:19.598522 1133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:19.602521 1133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:19.602576 1133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:19.602597 1133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:19.602613 1133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:19.602674 1133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:19.607853 1133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:39613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:39613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:19.608994 1133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:19.610025 1133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:19.619244 1151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:19.619424 1146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:19.619453 1133 server_base.cc:1034] running on GCE node
W20260807 18:42:19.619315 1147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:19.620070 1133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:19.621246 1133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:19.622452 1133 hybrid_clock.cc:648] HybridClock initialized: now 1786128139622421 us; error 47 us; skew 500 ppm
Aug 07 18:42:19 dist-test-slave-2d19 krb5kdc[1116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128139, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:19.625713 1133 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:19.626709 1133 webserver.cc:460] Webserver started at http://127.6.138.126:33557/ using document root <none> and password file <none>
I20260807 18:42:19.627315 1133 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:19.627390 1133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:19.627583 1133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:19.629894 1133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "bb2da04235bf4b31920b22856cc4a79a"
format_stamp: "Formatted at 2026-08-07 18:42:19 on dist-test-slave-2d19"
server_key: "e6fd1c6205ce7baec8b85705e8d5e426"
server_key_iv: "53acd4627a36dcd5f4497fbde2312ecb"
server_key_version: "encryptionkey@0"
I20260807 18:42:19.630494 1133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "bb2da04235bf4b31920b22856cc4a79a"
format_stamp: "Formatted at 2026-08-07 18:42:19 on dist-test-slave-2d19"
server_key: "e6fd1c6205ce7baec8b85705e8d5e426"
server_key_iv: "53acd4627a36dcd5f4497fbde2312ecb"
server_key_version: "encryptionkey@0"
I20260807 18:42:19.634680 1133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:42:19.637398 1164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:19.638702 1133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:42:19.638814 1133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "bb2da04235bf4b31920b22856cc4a79a"
format_stamp: "Formatted at 2026-08-07 18:42:19 on dist-test-slave-2d19"
server_key: "e6fd1c6205ce7baec8b85705e8d5e426"
server_key_iv: "53acd4627a36dcd5f4497fbde2312ecb"
server_key_version: "encryptionkey@0"
I20260807 18:42:19.638968 1133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:19.659456 1133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:19.662616 1133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:19.662822 1133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:19.670203 1133 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:39613
I20260807 18:42:19.670226 1247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:39613 every 8 connection(s)
I20260807 18:42:19.671173 1133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:19.674207 1248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:19.677877 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 1133
I20260807 18:42:19.677994 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:19.678323 6697 external_mini_cluster.cc:1428] Setting key ccd736482fe45184e2927d2fc2ffce0c
I20260807 18:42:19.680660 1248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a: Bootstrap starting.
I20260807 18:42:19.682899 1248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:19.683665 1248 log.cc:824] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:19.686094 1248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a: No bootstrap required, opened a new log
Aug 07 18:42:19 dist-test-slave-2d19 krb5kdc[1116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:19.690186 1248 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2da04235bf4b31920b22856cc4a79a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39613 } }
I20260807 18:42:19.690439 1248 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:19.690519 1248 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2da04235bf4b31920b22856cc4a79a, State: Initialized, Role: FOLLOWER
I20260807 18:42:19.691020 1248 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2da04235bf4b31920b22856cc4a79a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39613 } }
I20260807 18:42:19.691205 1248 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:19.691280 1248 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:19.691393 1248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:19.692387 1248 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2da04235bf4b31920b22856cc4a79a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39613 } }
I20260807 18:42:19.692760 1248 leader_election.cc:302] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb2da04235bf4b31920b22856cc4a79a; no voters:
I20260807 18:42:19.693142 1248 leader_election.cc:288] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:19.693284 1256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:19.693511 1256 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [term 1 LEADER]: Becoming Leader. State: Replica: bb2da04235bf4b31920b22856cc4a79a, State: Running, Role: LEADER
I20260807 18:42:19.693869 1256 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2da04235bf4b31920b22856cc4a79a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39613 } }
I20260807 18:42:19.694077 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:19.679804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45218 (local address 127.6.138.126:39613)
0807 18:42:19.680399 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:19.680417 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:19.680457 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:42:19.681162 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:19.681183 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:19.681522 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:19.681862 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:19.682706 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:19.683760 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:19.684994 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:19.685822 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:19.686014 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:19.688774 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:19.688780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:19.688794 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:19.688821 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:19.690698 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:19.691232 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:19.691237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:19.691241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:19.691463 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:19.691757 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:19.691761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:19.691763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:19.692217 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:19.692463 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:19.692733 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 18:42:19.693072 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":144,"threads_started":1}
I20260807 18:42:19.694440 1248 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:19.695662 1257 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb2da04235bf4b31920b22856cc4a79a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2da04235bf4b31920b22856cc4a79a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39613 } } }
I20260807 18:42:19.695811 1257 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:19.696200 1259 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb2da04235bf4b31920b22856cc4a79a. Latest consensus state: current_term: 1 leader_uuid: "bb2da04235bf4b31920b22856cc4a79a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb2da04235bf4b31920b22856cc4a79a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 39613 } } }
I20260807 18:42:19.696326 1259 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:19.696866 1268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:19.700212 1268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:19.706003 1268 catalog_manager.cc:1347] Generated new cluster ID: 6a6bc30b94274800a83943d396b64640
I20260807 18:42:19.706077 1268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:19.726639 1268 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:19.727713 1268 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:19.732290 1268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a: Generated new TSK 0
I20260807 18:42:19.732983 1268 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:42:19.763782 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39613
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:19.883515 1281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:19.883790 1281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:19.883838 1281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:19.887909 1281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:19.887979 1281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:19.888036 1281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:42:19.893437 1281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:19.894502 1281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:19.895680 1281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:42:19.906337 1281 server_base.cc:1034] running on GCE node
W20260807 18:42:19.906318 1293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:19.906297 1288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:19.906297 1290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:19.906984 1281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:19.907691 1281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:19.908893 1281 hybrid_clock.cc:648] HybridClock initialized: now 1786128139908884 us; error 35 us; skew 500 ppm
Aug 07 18:42:19 dist-test-slave-2d19 krb5kdc[1116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128139, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:19.911955 1281 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:42:19.912961 1281 webserver.cc:460] Webserver started at http://127.6.138.65:45099/ using document root <none> and password file <none>
I20260807 18:42:19.913606 1281 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:19.913707 1281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:19.913952 1281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:19.916083 1281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "17cb6589c19843af99e7b01bc165fd9a"
format_stamp: "Formatted at 2026-08-07 18:42:19 on dist-test-slave-2d19"
server_key: "c500e68a164c2fdf44fba068828c591d"
server_key_iv: "c4bc1912e161d250ac1a87ae1815cca6"
server_key_version: "encryptionkey@0"
I20260807 18:42:19.916723 1281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "17cb6589c19843af99e7b01bc165fd9a"
format_stamp: "Formatted at 2026-08-07 18:42:19 on dist-test-slave-2d19"
server_key: "c500e68a164c2fdf44fba068828c591d"
server_key_iv: "c4bc1912e161d250ac1a87ae1815cca6"
server_key_version: "encryptionkey@0"
I20260807 18:42:19.920622 1281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:42:19.923264 1304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:19.925002 1281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:42:19.925170 1281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "17cb6589c19843af99e7b01bc165fd9a"
format_stamp: "Formatted at 2026-08-07 18:42:19 on dist-test-slave-2d19"
server_key: "c500e68a164c2fdf44fba068828c591d"
server_key_iv: "c4bc1912e161d250ac1a87ae1815cca6"
server_key_version: "encryptionkey@0"
I20260807 18:42:19.925303 1281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:19.943583 1281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:19.947060 1281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:19.947278 1281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:19.947911 1281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:19.948854 1281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:19.948928 1281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:19.949002 1281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:19.949059 1281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:19.958737 1281 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:39203
I20260807 18:42:19.958763 1477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:39203 every 8 connection(s)
I20260807 18:42:19.959708 1281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:42:19.961755 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 1281
I20260807 18:42:19.961930 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:42:19.962285 6697 external_mini_cluster.cc:1428] Setting key ef2acca03c6605f56ed18a42a8a67337
Aug 07 18:42:19 dist-test-slave-2d19 krb5kdc[1116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128139, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:42:19.975687 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:19.961679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:57953 (local address 127.6.138.126:39613)
0807 18:42:19.961849 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:19.961855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:19.962800 (+ 945us) server_negotiation.cc:407] Connection header received
0807 18:42:19.964608 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:19.964614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:19.964680 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:19.964780 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:19.966242 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:19.966770 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:19.968677 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:19.968914 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:19.968998 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:19.971988 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:19.971992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:19.971995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:19.972027 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:19.973553 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:19.974110 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:19.974114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:19.974116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:19.974200 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:19.974557 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:19.974561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:19.974563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:19.974792 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:19.974902 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:19.975423 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 18:42:19.975557 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:42:19.976624 1481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:19.962104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39613 (local address 127.6.138.65:57953)
0807 18:42:19.962617 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:19.962671 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:19.964377 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:19.964988 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:19.964995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:19.965369 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:19.966064 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:19.966091 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:19.966920 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:19.966926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:19.968519 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:19.968534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:19.969084 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:19.969092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:19.969339 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:19.970005 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:19.970032 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:19.971798 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:19.973684 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:19.973692 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:19.973710 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:19.973988 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:19.974299 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:19.974302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:19.974305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:19.974450 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:19.974915 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:19.974922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:19.975181 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 18:42:19.975357 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 18:42:19.975654 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
I20260807 18:42:19.978015 1478 heartbeater.cc:347] Connected to a master server at 127.6.138.126:39613
I20260807 18:42:19.978296 1478 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:19.978920 1478 heartbeater.cc:511] Master 127.6.138.126:39613 requested a full tablet report, sending...
I20260807 18:42:19.980825 1193 ts_manager.cc:194] Registered new tserver with Master: 17cb6589c19843af99e7b01bc165fd9a (127.6.138.65:39203)
I20260807 18:42:19.981972 1193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:57953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:42:19.996143 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39613
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:20.112726 1486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:20.113003 1486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:20.113082 1486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:20.117146 1486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:20.117214 1486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:20.117318 1486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:42:20.122736 1486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:20.123867 1486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:20.124985 1486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:20.135118 1496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:20.135149 1492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:20.135118 1494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:20.135475 1486 server_base.cc:1034] running on GCE node
I20260807 18:42:20.136018 1486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:20.136838 1486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:20.138033 1486 hybrid_clock.cc:648] HybridClock initialized: now 1786128140138018 us; error 27 us; skew 500 ppm
Aug 07 18:42:20 dist-test-slave-2d19 krb5kdc[1116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128140, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:20.141000 1486 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:42:20.141889 1486 webserver.cc:460] Webserver started at http://127.6.138.66:40783/ using document root <none> and password file <none>
I20260807 18:42:20.142540 1486 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:20.142639 1486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:20.142956 1486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:20.145221 1486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "b27cd1e651e6455b8c8c6cf0f42fb197"
format_stamp: "Formatted at 2026-08-07 18:42:20 on dist-test-slave-2d19"
server_key: "2ac8ecd6a9974a79fc9c72469678b1ba"
server_key_iv: "cca40582a9e252f40cdd2f73d85f10c7"
server_key_version: "encryptionkey@0"
I20260807 18:42:20.145833 1486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "b27cd1e651e6455b8c8c6cf0f42fb197"
format_stamp: "Formatted at 2026-08-07 18:42:20 on dist-test-slave-2d19"
server_key: "2ac8ecd6a9974a79fc9c72469678b1ba"
server_key_iv: "cca40582a9e252f40cdd2f73d85f10c7"
server_key_version: "encryptionkey@0"
I20260807 18:42:20.149739 1486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:42:20.152357 1506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:20.153622 1486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:20.153780 1486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "b27cd1e651e6455b8c8c6cf0f42fb197"
format_stamp: "Formatted at 2026-08-07 18:42:20 on dist-test-slave-2d19"
server_key: "2ac8ecd6a9974a79fc9c72469678b1ba"
server_key_iv: "cca40582a9e252f40cdd2f73d85f10c7"
server_key_version: "encryptionkey@0"
I20260807 18:42:20.153916 1486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:20.185598 1486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:20.189174 1486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:20.189400 1486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:20.190042 1486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:20.190917 1486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:20.190989 1486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:20.191063 1486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:20.191123 1486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:20.200800 1486 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:40237
I20260807 18:42:20.200822 1672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:40237 every 8 connection(s)
I20260807 18:42:20.201733 1486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:42:20.204449 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 1486
I20260807 18:42:20.204610 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:42:20.205014 6697 external_mini_cluster.cc:1428] Setting key 00e2c6fc83bd6053d6b6586cbc529b90
Aug 07 18:42:20 dist-test-slave-2d19 krb5kdc[1116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128140, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:42:20.217218 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.203636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:60335 (local address 127.6.138.126:39613)
0807 18:42:20.203782 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.203785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.204808 (+ 1023us) server_negotiation.cc:407] Connection header received
0807 18:42:20.205983 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.205986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.206031 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.206122 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:20.207314 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.208156 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.209950 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.210209 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.210299 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.213166 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:20.213169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:20.213171 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:20.213197 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:20.214875 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:20.215447 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:20.215457 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:20.215460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:20.215561 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:20.216019 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:20.216022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:20.216023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:20.216294 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:20.216384 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.216978 (+ 594us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.217084 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:42:20.218185 1677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.204025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39613 (local address 127.6.138.66:60335)
0807 18:42:20.204559 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:20.204642 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:20.205785 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:20.206282 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:20.206288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:20.206682 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:20.207161 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.207174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.208347 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.208351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.209808 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.209823 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.210377 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.210380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.210646 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.211312 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:20.211329 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:20.212976 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:20.215030 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:20.215037 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:20.215053 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:20.215329 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:20.215690 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:20.215694 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:20.215696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:20.215885 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:20.216420 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:20.216431 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:20.216709 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 18:42:20.216905 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 18:42:20.217173 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":152,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
I20260807 18:42:20.219539 1674 heartbeater.cc:347] Connected to a master server at 127.6.138.126:39613
I20260807 18:42:20.219817 1674 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:20.220410 1674 heartbeater.cc:511] Master 127.6.138.126:39613 requested a full tablet report, sending...
I20260807 18:42:20.221585 1193 ts_manager.cc:194] Registered new tserver with Master: b27cd1e651e6455b8c8c6cf0f42fb197 (127.6.138.66:40237)
I20260807 18:42:20.222479 1193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:60335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:42:20.237531 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39613
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:40027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:20.356971 1680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:20.357345 1680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:20.357429 1680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:20.361657 1680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:20.361719 1680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:20.361805 1680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:42:20.367142 1680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:40027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:39613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:20.368306 1680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:20.369403 1680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:20.379492 1690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:20.379474 1686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:20.379714 1680 server_base.cc:1034] running on GCE node
W20260807 18:42:20.379467 1687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:20.380319 1680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:20.380992 1680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:20.382159 1680 hybrid_clock.cc:648] HybridClock initialized: now 1786128140382139 us; error 33 us; skew 500 ppm
Aug 07 18:42:20 dist-test-slave-2d19 krb5kdc[1116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128140, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:20.385367 1680 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:42:20.386296 1680 webserver.cc:460] Webserver started at http://127.6.138.67:38489/ using document root <none> and password file <none>
I20260807 18:42:20.386881 1680 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:20.386945 1680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:20.387171 1680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:20.389310 1680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19"
format_stamp: "Formatted at 2026-08-07 18:42:20 on dist-test-slave-2d19"
server_key: "b221180954c5a8c913ac880aad828b0b"
server_key_iv: "27853c89a4646dac293c3a69eeaa7dbf"
server_key_version: "encryptionkey@0"
I20260807 18:42:20.389901 1680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19"
format_stamp: "Formatted at 2026-08-07 18:42:20 on dist-test-slave-2d19"
server_key: "b221180954c5a8c913ac880aad828b0b"
server_key_iv: "27853c89a4646dac293c3a69eeaa7dbf"
server_key_version: "encryptionkey@0"
I20260807 18:42:20.393893 1680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:42:20.396802 1700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:20.398098 1680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:42:20.398222 1680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19"
format_stamp: "Formatted at 2026-08-07 18:42:20 on dist-test-slave-2d19"
server_key: "b221180954c5a8c913ac880aad828b0b"
server_key_iv: "27853c89a4646dac293c3a69eeaa7dbf"
server_key_version: "encryptionkey@0"
I20260807 18:42:20.398341 1680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:20.427762 1680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:20.431421 1680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:20.431643 1680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:20.432350 1680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:20.433220 1680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:20.433269 1680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:20.433339 1680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:20.433382 1680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:20.443269 1680 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:43739
I20260807 18:42:20.443291 1859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:43739 every 8 connection(s)
I20260807 18:42:20.444281 1680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:42:20.445935 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 1680
I20260807 18:42:20.446120 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:42:20.446521 6697 external_mini_cluster.cc:1428] Setting key 980b32237eef82e33986a22087a8a121
Aug 07 18:42:20 dist-test-slave-2d19 krb5kdc[1116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128140, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:20.460426 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.446249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:51183 (local address 127.6.138.126:39613)
0807 18:42:20.446413 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.446418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.447479 (+ 1061us) server_negotiation.cc:407] Connection header received
0807 18:42:20.449558 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.449562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.449623 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.449721 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:20.451036 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.451633 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.453208 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.453462 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.453587 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.456331 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:20.456334 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:20.456337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:20.456371 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:20.457885 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:20.458562 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:20.458568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:20.458570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:20.458683 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:20.459079 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:20.459083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:20.459085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:20.459289 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:20.459412 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.460069 (+ 657us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.460238 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:42:20.461489 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.446728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:39613 (local address 127.6.138.67:51183)
0807 18:42:20.447292 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:20.447367 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:20.449312 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:20.449914 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:20.449923 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:20.450344 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:20.450873 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.450889 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.451788 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.451792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.453063 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.453075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.453637 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.453641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.453835 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.454486 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:20.454504 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:20.456107 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:20.458090 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:20.458097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:20.458112 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:20.458433 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:20.458788 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:20.458792 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:20.458794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:20.458967 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:20.459413 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:20.459431 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:20.459694 (+ 263us) client_negotiation.cc:769] Sending connection context
0807 18:42:20.459928 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 18:42:20.460387 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":120,"threads_started":1}
I20260807 18:42:20.462667 1860 heartbeater.cc:347] Connected to a master server at 127.6.138.126:39613
I20260807 18:42:20.462903 1860 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:20.463389 1860 heartbeater.cc:511] Master 127.6.138.126:39613 requested a full tablet report, sending...
I20260807 18:42:20.464480 1193 ts_manager.cc:194] Registered new tserver with Master: 8cfd6734c47d4ebeb6a46ee626f1ac19 (127.6.138.67:43739)
I20260807 18:42:20.465085 1193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:51183
I20260807 18:42:20.473212 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:42:20.483752 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.475311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.6.138.126:39613)
0807 18:42:20.475462 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.475467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.475699 (+ 232us) server_negotiation.cc:407] Connection header received
0807 18:42:20.475791 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.475794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.475834 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.475901 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:20.476691 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.477222 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.478095 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.478308 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.478397 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.479936 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:20.479939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:20.479941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:20.479966 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:20.481705 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:20.482275 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:20.482278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:20.482279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:20.482360 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:20.482763 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:20.482766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:20.482767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:20.482997 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:20.483121 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.483396 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.483537 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:42:20.487106 1193 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:42:20.490263 1193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:42:20.505764 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.499928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43739 (local address 127.0.0.1:35392)
0807 18:42:20.500628 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:20.500642 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:20.500725 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:20.501497 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:20.501500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:20.501525 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:20.501599 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.501607 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.502972 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.502975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.504287 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.504299 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.505188 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.505192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.505300 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.505369 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 18:42:20.505464 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:42:20.505598 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":80,"thread_start_us":116,"threads_started":1}
I20260807 18:42:20.506588 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.500422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.6.138.67:43739)
0807 18:42:20.500992 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.501015 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.501038 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:42:20.501122 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.501127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.501317 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.501522 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:20.501716 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.502834 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.504419 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.505020 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.506084 (+ 1064us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.506253 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.506313 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.506425 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":135,"threads_started":1}
I20260807 18:42:20.506647 1874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.499216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:39203 (local address 127.0.0.1:36804)
0807 18:42:20.499854 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:20.499911 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:20.500049 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:20.501133 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:20.501139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:20.501163 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:20.501304 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.501311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.502888 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.502894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.504972 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.504984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.506240 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.506244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.506335 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.506360 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:42:20.506434 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:42:20.506500 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260807 18:42:20.507330 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.499216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:40237 (local address 127.0.0.1:47740)
0807 18:42:20.499933 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:20.499948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:20.500048 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:20.501294 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:20.501297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:20.501316 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:20.501418 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.501424 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.502675 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.502681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.504295 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.504308 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.506965 (+ 2657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.506969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.507056 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.507079 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:42:20.507151 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 18:42:20.507214 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":262,"thread_start_us":103,"threads_started":1}
I20260807 18:42:20.507555 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.499394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47740 (local address 127.6.138.66:40237)
0807 18:42:20.500854 (+ 1460us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.500870 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.500896 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:42:20.500982 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.500986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.501141 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.501312 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:20.501568 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.502537 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.504448 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.506828 (+ 2380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.506946 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.507081 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.507317 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.507405 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":174,"threads_started":1}
I20260807 18:42:20.507558 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.499947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.6.138.65:39203)
0807 18:42:20.500436 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.500443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.500467 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:42:20.500551 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.500556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.500788 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.500955 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:20.501457 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.502712 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.505117 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.505903 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.507101 (+ 1198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.507243 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.507297 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.507402 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":138,"threads_started":1}
I20260807 18:42:20.509963 1771 tablet_service.cc:1467] Processing CreateTablet for tablet 66b3e0098557472eb148f6ef37b7b30b (DEFAULT_TABLE table=test-table [id=40cfa4143bd34d5ea99395ae984aa555]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:20.510308 1385 tablet_service.cc:1467] Processing CreateTablet for tablet 66b3e0098557472eb148f6ef37b7b30b (DEFAULT_TABLE table=test-table [id=40cfa4143bd34d5ea99395ae984aa555]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:20.511085 1771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66b3e0098557472eb148f6ef37b7b30b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:20.510838 1587 tablet_service.cc:1467] Processing CreateTablet for tablet 66b3e0098557472eb148f6ef37b7b30b (DEFAULT_TABLE table=test-table [id=40cfa4143bd34d5ea99395ae984aa555]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:20.511421 1385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66b3e0098557472eb148f6ef37b7b30b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:20.511904 1587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66b3e0098557472eb148f6ef37b7b30b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:20.517853 1880 tablet_bootstrap.cc:492] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: Bootstrap starting.
I20260807 18:42:20.519265 1881 tablet_bootstrap.cc:492] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a: Bootstrap starting.
I20260807 18:42:20.519860 1884 tablet_bootstrap.cc:492] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197: Bootstrap starting.
I20260807 18:42:20.520342 1880 tablet_bootstrap.cc:654] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:20.521512 1881 tablet_bootstrap.cc:654] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:20.521626 1880 log.cc:824] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:20.521984 1884 tablet_bootstrap.cc:654] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:20.522562 1881 log.cc:824] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:20.523209 1884 log.cc:824] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:20.523862 1880 tablet_bootstrap.cc:492] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: No bootstrap required, opened a new log
I20260807 18:42:20.524083 1880 ts_tablet_manager.cc:1397] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 18:42:20.524684 1881 tablet_bootstrap.cc:492] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a: No bootstrap required, opened a new log
I20260807 18:42:20.524989 1881 ts_tablet_manager.cc:1397] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:42:20.525282 1884 tablet_bootstrap.cc:492] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197: No bootstrap required, opened a new log
I20260807 18:42:20.525475 1884 ts_tablet_manager.cc:1397] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:42:20.527921 1880 raft_consensus.cc:348] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } }
I20260807 18:42:20.528220 1880 raft_consensus.cc:374] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:20.528316 1880 raft_consensus.cc:729] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfd6734c47d4ebeb6a46ee626f1ac19, State: Initialized, Role: FOLLOWER
I20260807 18:42:20.528867 1880 consensus_queue.cc:260] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } }
I20260807 18:42:20.529596 1860 heartbeater.cc:503] Master 127.6.138.126:39613 was elected leader, sending a full tablet report...
I20260807 18:42:20.529651 1884 raft_consensus.cc:348] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } }
I20260807 18:42:20.529932 1884 raft_consensus.cc:374] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:20.529991 1884 raft_consensus.cc:729] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27cd1e651e6455b8c8c6cf0f42fb197, State: Initialized, Role: FOLLOWER
I20260807 18:42:20.530122 1880 ts_tablet_manager.cc:1428] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 18:42:20.530496 1884 consensus_queue.cc:260] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } }
I20260807 18:42:20.530901 1881 raft_consensus.cc:348] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } }
I20260807 18:42:20.531163 1881 raft_consensus.cc:374] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:20.531224 1881 raft_consensus.cc:729] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cb6589c19843af99e7b01bc165fd9a, State: Initialized, Role: FOLLOWER
I20260807 18:42:20.531519 1884 ts_tablet_manager.cc:1428] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:42:20.531670 1881 consensus_queue.cc:260] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } }
I20260807 18:42:20.531915 1674 heartbeater.cc:503] Master 127.6.138.126:39613 was elected leader, sending a full tablet report...
I20260807 18:42:20.532441 1478 heartbeater.cc:503] Master 127.6.138.126:39613 was elected leader, sending a full tablet report...
I20260807 18:42:20.532660 1881 ts_tablet_manager.cc:1428] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260807 18:42:20.593458 1891 raft_consensus.cc:482] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:20.593838 1891 raft_consensus.cc:504] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } }
I20260807 18:42:20.595293 1891 leader_election.cc:288] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b27cd1e651e6455b8c8c6cf0f42fb197 (127.6.138.66:40237), 8cfd6734c47d4ebeb6a46ee626f1ac19 (127.6.138.67:43739)
I20260807 18:42:20.599474 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.595766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:49427 (local address 127.6.138.67:43739)
0807 18:42:20.595934 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.595940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.596188 (+ 248us) server_negotiation.cc:407] Connection header received
0807 18:42:20.596374 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.596378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.596430 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.596518 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:20.596822 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.597351 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.598461 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.598901 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.599037 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.599092 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.599281 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.599351 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:42:20.599506 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.595586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43739 (local address 127.6.138.65:49427)
0807 18:42:20.596059 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:20.596077 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:20.596245 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:20.596534 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:20.596537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:20.596555 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:20.596702 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.596709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.597468 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.597473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.598328 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.598338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.599035 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.599039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.599136 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.599160 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:42:20.599261 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:42:20.599336 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":23,"thread_start_us":533,"threads_started":1}
I20260807 18:42:20.599658 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.595586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:40237 (local address 127.6.138.65:60063)
0807 18:42:20.596357 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:20.596370 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:20.596513 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:20.596800 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:20.596803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:20.596819 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:20.596946 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.596952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.597688 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.597692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.598487 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:20.598496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.599188 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.599191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:20.599308 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.599332 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:42:20.599428 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:42:20.599508 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":28,"thread_start_us":211,"threads_started":1}
I20260807 18:42:20.599637 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.595841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:60063 (local address 127.6.138.66:40237)
0807 18:42:20.596010 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.596020 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.596469 (+ 449us) server_negotiation.cc:407] Connection header received
0807 18:42:20.596638 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.596641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.596695 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.596777 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:20.597085 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.597576 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.598628 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.599059 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.599193 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.599248 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.599447 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.599516 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:42:20.600396 1610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66b3e0098557472eb148f6ef37b7b30b" candidate_uuid: "17cb6589c19843af99e7b01bc165fd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" is_pre_election: true
I20260807 18:42:20.600582 1794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66b3e0098557472eb148f6ef37b7b30b" candidate_uuid: "17cb6589c19843af99e7b01bc165fd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" is_pre_election: true
I20260807 18:42:20.600723 1610 raft_consensus.cc:2454] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cb6589c19843af99e7b01bc165fd9a in term 0.
I20260807 18:42:20.600910 1794 raft_consensus.cc:2454] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cb6589c19843af99e7b01bc165fd9a in term 0.
I20260807 18:42:20.601270 1309 leader_election.cc:302] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cb6589c19843af99e7b01bc165fd9a, b27cd1e651e6455b8c8c6cf0f42fb197; no voters:
I20260807 18:42:20.601573 1891 raft_consensus.cc:2781] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:42:20.601661 1891 raft_consensus.cc:482] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:42:20.601692 1891 raft_consensus.cc:3037] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:20.603029 1891 raft_consensus.cc:504] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } }
I20260807 18:42:20.603403 1891 leader_election.cc:288] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [CANDIDATE]: Term 1 election: Requested vote from peers b27cd1e651e6455b8c8c6cf0f42fb197 (127.6.138.66:40237), 8cfd6734c47d4ebeb6a46ee626f1ac19 (127.6.138.67:43739)
I20260807 18:42:20.603811 1794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66b3e0098557472eb148f6ef37b7b30b" candidate_uuid: "17cb6589c19843af99e7b01bc165fd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19"
I20260807 18:42:20.603823 1610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66b3e0098557472eb148f6ef37b7b30b" candidate_uuid: "17cb6589c19843af99e7b01bc165fd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197"
I20260807 18:42:20.603951 1610 raft_consensus.cc:3037] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:20.603951 1794 raft_consensus.cc:3037] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:20.605331 1610 raft_consensus.cc:2454] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17cb6589c19843af99e7b01bc165fd9a in term 1.
I20260807 18:42:20.605330 1794 raft_consensus.cc:2454] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17cb6589c19843af99e7b01bc165fd9a in term 1.
I20260807 18:42:20.605695 1308 leader_election.cc:302] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cb6589c19843af99e7b01bc165fd9a, 8cfd6734c47d4ebeb6a46ee626f1ac19; no voters:
I20260807 18:42:20.605902 1891 raft_consensus.cc:2781] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:20.606149 1891 raft_consensus.cc:686] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 1 LEADER]: Becoming Leader. State: Replica: 17cb6589c19843af99e7b01bc165fd9a, State: Running, Role: LEADER
I20260807 18:42:20.606491 1891 consensus_queue.cc:237] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } }
I20260807 18:42:20.609658 1192 catalog_manager.cc:5495] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 17cb6589c19843af99e7b01bc165fd9a (127.6.138.65). New cstate: current_term: 1 leader_uuid: "17cb6589c19843af99e7b01bc165fd9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17cb6589c19843af99e7b01bc165fd9a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 39203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 } health_report { overall_health: UNKNOWN } } }
I20260807 18:42:20.635974 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.632394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.6.138.65:39203)
0807 18:42:20.632550 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.632554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.632571 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:42:20.632655 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.632657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.632716 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.632840 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:42:20.633108 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.633676 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.634531 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.634763 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.634846 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.635152 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:42:20.635620 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:42:20.635721 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.635793 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.635850 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:42:20.642383 1794 raft_consensus.cc:1261] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [term 1 FOLLOWER]: Refusing update from remote peer 17cb6589c19843af99e7b01bc165fd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:20.642383 1610 raft_consensus.cc:1261] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [term 1 FOLLOWER]: Refusing update from remote peer 17cb6589c19843af99e7b01bc165fd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:20.643222 1891 consensus_queue.cc:1035] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cfd6734c47d4ebeb6a46ee626f1ac19" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:20.643448 1894 consensus_queue.cc:1035] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b27cd1e651e6455b8c8c6cf0f42fb197" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:20.651260 1899 mvcc.cc:205] Tried to move back new op lower bound from 7315980864063262720 to 7315980863926796288. Current Snapshot: MvccSnapshot[applied={T|T < 7315980864063262720}]
I20260807 18:42:20.654891 1902 mvcc.cc:205] Tried to move back new op lower bound from 7315980864063262720 to 7315980863926796288. Current Snapshot: MvccSnapshot[applied={T|T < 7315980864063262720}]
I20260807 18:42:20.660744 1192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:20.660905 1192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:20.663492 1192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb2da04235bf4b31920b22856cc4a79a: Sending DeleteTablet for 3 replicas of tablet 66b3e0098557472eb148f6ef37b7b30b
I20260807 18:42:20.664326 1385 tablet_service.cc:1514] Processing DeleteTablet for tablet 66b3e0098557472eb148f6ef37b7b30b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:20 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:36804
I20260807 18:42:20.664410 1587 tablet_service.cc:1514] Processing DeleteTablet for tablet 66b3e0098557472eb148f6ef37b7b30b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:20 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:47740
I20260807 18:42:20.664489 1771 tablet_service.cc:1514] Processing DeleteTablet for tablet 66b3e0098557472eb148f6ef37b7b30b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:20 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:35392
I20260807 18:42:20.664942 1914 tablet_replica.cc:330] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: stopping tablet replica
I20260807 18:42:20.665067 1913 tablet_replica.cc:330] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a: stopping tablet replica
I20260807 18:42:20.665351 1914 raft_consensus.cc:2229] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:42:20.665388 1913 raft_consensus.cc:2229] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:42:20.665740 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 1281
I20260807 18:42:20.665793 1913 raft_consensus.cc:2258] T 66b3e0098557472eb148f6ef37b7b30b P 17cb6589c19843af99e7b01bc165fd9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:20.665987 1914 raft_consensus.cc:2258] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:20.666320 1912 tablet_replica.cc:330] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197: stopping tablet replica
I20260807 18:42:20.668948 1912 raft_consensus.cc:2229] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:42:20.669245 1912 raft_consensus.cc:2258] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:20.671841 1914 ts_tablet_manager.cc:1905] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:20.675494 1914 ts_tablet_manager.cc:1918] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:20.675632 1914 log.cc:1196] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/66b3e0098557472eb148f6ef37b7b30b
I20260807 18:42:20.676069 1914 ts_tablet_manager.cc:1939] T 66b3e0098557472eb148f6ef37b7b30b P 8cfd6734c47d4ebeb6a46ee626f1ac19: Deleting consensus metadata
I20260807 18:42:20.677522 1171 catalog_manager.cc:4841] TS 8cfd6734c47d4ebeb6a46ee626f1ac19 (127.6.138.67:43739): tablet 66b3e0098557472eb148f6ef37b7b30b (table test-table [id=40cfa4143bd34d5ea99395ae984aa555]) successfully deleted
I20260807 18:42:20.678231 1912 ts_tablet_manager.cc:1905] T 66b3e0098557472eb148f6ef37b7b30b P b27cd1e651e6455b8c8c6cf0f42fb197: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 18:42:20.680554 1166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:39203 (error 108)
I20260807 18:42:20.680784 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 1486
I20260807 18:42:20.687546 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.687116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:39203 (local address 127.0.0.1:36822)
0807 18:42:20.687310 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:20.687402 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:39203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 18:42:20.687767 1166 catalog_manager.cc:4584] TS 17cb6589c19843af99e7b01bc165fd9a (127.6.138.65:39203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 66b3e0098557472eb148f6ef37b7b30b: Network error: Client connection negotiation failed: client connection to 127.6.138.65:39203: connect: Connection refused (error 111)
I20260807 18:42:20.692723 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 1680
I20260807 18:42:20.700711 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.692851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:40237 (local address 127.0.0.1:47746)
0807 18:42:20.700497 (+ 7646us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:20.700585 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.66:40237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7500}
I20260807 18:42:20.706390 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 1133
2026-08-07T18:42:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2631 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T18:42:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:20Z Disabled control of system clock
I20260807 18:42:20.746946 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:34491
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:46253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:34491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:42:20.866508 1923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:20.866775 1923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:20.866811 1923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:20.870895 1923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:20.870959 1923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:20.870975 1923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:20.870991 1923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:20.871052 1923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:20.875895 1923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:34491
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:34491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:20.877063 1923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:20.878471 1923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:20.887432 1933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:20.887497 1932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:20.887498 1935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:20.887537 1923 server_base.cc:1034] running on GCE node
I20260807 18:42:20.888412 1923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:20.889565 1923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:20.890754 1923 hybrid_clock.cc:648] HybridClock initialized: now 1786128140890735 us; error 31 us; skew 500 ppm
I20260807 18:42:20.892728 1923 webserver.cc:460] Webserver started at http://127.6.138.126:37891/ using document root <none> and password file <none>
I20260807 18:42:20.893318 1923 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:20.893416 1923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:20.893659 1923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:20.895717 1923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "8a92d62c8a474fa9bf8994f4d006c308"
format_stamp: "Formatted at 2026-08-07 18:42:20 on dist-test-slave-2d19"
server_key: "03de9e90d44e30a3ac301b82a2c784ed"
server_key_iv: "07a89d78e68a2b30c19c31e8d9c6c463"
server_key_version: "encryptionkey@0"
I20260807 18:42:20.896350 1923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "8a92d62c8a474fa9bf8994f4d006c308"
format_stamp: "Formatted at 2026-08-07 18:42:20 on dist-test-slave-2d19"
server_key: "03de9e90d44e30a3ac301b82a2c784ed"
server_key_iv: "07a89d78e68a2b30c19c31e8d9c6c463"
server_key_version: "encryptionkey@0"
I20260807 18:42:20.900143 1923 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:42:20.903009 1947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:20.904402 1923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:42:20.904569 1923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "8a92d62c8a474fa9bf8994f4d006c308"
format_stamp: "Formatted at 2026-08-07 18:42:20 on dist-test-slave-2d19"
server_key: "03de9e90d44e30a3ac301b82a2c784ed"
server_key_iv: "07a89d78e68a2b30c19c31e8d9c6c463"
server_key_version: "encryptionkey@0"
I20260807 18:42:20.904699 1923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:20.936065 1923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:20.936975 1923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:20.937224 1923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:20.945322 1923 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:34491
I20260807 18:42:20.945330 2046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:34491 every 8 connection(s)
I20260807 18:42:20.946409 1923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:20.949781 2050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:20.953840 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 1923
I20260807 18:42:20.954006 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:20.954311 6697 external_mini_cluster.cc:1428] Setting key 29f4b4bafe641a89861a31a888edaec7
I20260807 18:42:20.955752 2050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308: Bootstrap starting.
I20260807 18:42:20.958108 2050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:20.958950 2050 log.cc:824] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:20.961167 2050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308: No bootstrap required, opened a new log
I20260807 18:42:20.964381 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:20.955971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58958 (local address 127.6.138.126:34491)
0807 18:42:20.956682 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:20.956700 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:20.956739 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:42:20.957419 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:20.957437 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:20.957853 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:20.958158 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:20.958613 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.959730 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.960930 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:20.961530 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:20.961662 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:20.962550 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:20.962556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:20.962569 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:20.962943 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:20.963021 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:20.963043 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:20.963124 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:20.963299 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 18:42:20.963599 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":151,"threads_started":1}
I20260807 18:42:20.965225 2050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a92d62c8a474fa9bf8994f4d006c308" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34491 } }
I20260807 18:42:20.965436 2050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:20.965521 2050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a92d62c8a474fa9bf8994f4d006c308, State: Initialized, Role: FOLLOWER
I20260807 18:42:20.965983 2050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a92d62c8a474fa9bf8994f4d006c308" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34491 } }
I20260807 18:42:20.966107 2050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:20.966193 2050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:20.966305 2050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:20.967273 2050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a92d62c8a474fa9bf8994f4d006c308" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34491 } }
I20260807 18:42:20.967610 2050 leader_election.cc:302] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a92d62c8a474fa9bf8994f4d006c308; no voters:
I20260807 18:42:20.967905 2050 leader_election.cc:288] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:20.968036 2055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:20.968313 2055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [term 1 LEADER]: Becoming Leader. State: Replica: 8a92d62c8a474fa9bf8994f4d006c308, State: Running, Role: LEADER
I20260807 18:42:20.968640 2055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a92d62c8a474fa9bf8994f4d006c308" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34491 } }
I20260807 18:42:20.969102 2050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:20.970144 2057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a92d62c8a474fa9bf8994f4d006c308. Latest consensus state: current_term: 1 leader_uuid: "8a92d62c8a474fa9bf8994f4d006c308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a92d62c8a474fa9bf8994f4d006c308" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34491 } } }
I20260807 18:42:20.970274 2057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:20.970798 2056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a92d62c8a474fa9bf8994f4d006c308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a92d62c8a474fa9bf8994f4d006c308" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 34491 } } }
I20260807 18:42:20.970968 2056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308 [sys.catalog]: This master's current role is: LEADER
W20260807 18:42:20.974372 2080 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:42:20.974466 2080 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:42:20.974527 2069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:20.975303 2069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:20.981000 2069 catalog_manager.cc:1347] Generated new cluster ID: 9a589314891b4113b4f46cbc1af4dcc6
I20260807 18:42:20.981068 2069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:20.997040 2069 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:20.998329 2069 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:21.005668 2069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308: Generated new TSK 0
I20260807 18:42:21.006518 2069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:21.020398 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34491
--builtin_ntp_servers=127.6.138.84:46253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:42:21.140017 2086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:21.140388 2086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:21.140465 2086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:21.144563 2086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:21.144654 2086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:21.144748 2086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:42:21.149752 2086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:21.150978 2086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:21.152539 2086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:21.162225 2099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:21.162233 2098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:21.162346 2086 server_base.cc:1034] running on GCE node
W20260807 18:42:21.162233 2107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:21.162879 2086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:21.163563 2086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:21.164726 2086 hybrid_clock.cc:648] HybridClock initialized: now 1786128141164709 us; error 29 us; skew 500 ppm
I20260807 18:42:21.166615 2086 webserver.cc:460] Webserver started at http://127.6.138.65:37971/ using document root <none> and password file <none>
I20260807 18:42:21.167208 2086 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:21.167276 2086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:21.167491 2086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:21.169773 2086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "15f4668220114143ad73c5726a42be41"
format_stamp: "Formatted at 2026-08-07 18:42:21 on dist-test-slave-2d19"
server_key: "7fc39149e66faf94a3e53f6a7d4b4b3a"
server_key_iv: "2a2abe84b7f36bf3afd5622a217d1cd1"
server_key_version: "encryptionkey@0"
I20260807 18:42:21.170401 2086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "15f4668220114143ad73c5726a42be41"
format_stamp: "Formatted at 2026-08-07 18:42:21 on dist-test-slave-2d19"
server_key: "7fc39149e66faf94a3e53f6a7d4b4b3a"
server_key_iv: "2a2abe84b7f36bf3afd5622a217d1cd1"
server_key_version: "encryptionkey@0"
I20260807 18:42:21.174237 2086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 18:42:21.177042 2114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:21.178486 2086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:42:21.178651 2086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "15f4668220114143ad73c5726a42be41"
format_stamp: "Formatted at 2026-08-07 18:42:21 on dist-test-slave-2d19"
server_key: "7fc39149e66faf94a3e53f6a7d4b4b3a"
server_key_iv: "2a2abe84b7f36bf3afd5622a217d1cd1"
server_key_version: "encryptionkey@0"
I20260807 18:42:21.178785 2086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:21.196195 2086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:21.196985 2086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:21.197193 2086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:21.197817 2086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:21.198803 2086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:21.198877 2086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:21.198946 2086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:21.198994 2086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:21.207929 2086 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:40139
I20260807 18:42:21.207952 2286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:40139 every 8 connection(s)
I20260807 18:42:21.208964 2086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:42:21.218243 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.210886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:48797 (local address 127.6.138.126:34491)
0807 18:42:21.211033 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.211037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.212137 (+ 1100us) server_negotiation.cc:407] Connection header received
0807 18:42:21.213554 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.213557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.213605 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.213685 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:21.214581 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.215287 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.216480 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.216704 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.216791 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.217826 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:21.217829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:21.217831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:21.217843 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:21.217862 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:21.217869 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:21.217930 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.218076 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.218127 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:42:21.218618 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 2086
I20260807 18:42:21.218755 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:42:21.219084 6697 external_mini_cluster.cc:1428] Setting key 55e9bb63cc4585be89cf154057616110
I20260807 18:42:21.219336 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.211316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:34491 (local address 127.6.138.65:48797)
0807 18:42:21.211980 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:21.212021 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:21.213320 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:21.213853 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:21.213860 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:21.213942 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:21.214412 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.214424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.215409 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.215411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.216367 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.216377 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.216848 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.216850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.216990 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.217574 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:42:21.217576 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:21.217616 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:42:21.217678 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:21.217946 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:21.217953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:21.217958 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:42:21.218053 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:42:21.218269 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":128,"threads_started":1}
I20260807 18:42:21.220636 2287 heartbeater.cc:347] Connected to a master server at 127.6.138.126:34491
I20260807 18:42:21.220878 2287 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:21.221223 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34491
--builtin_ntp_servers=127.6.138.84:46253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:42:21.221513 2287 heartbeater.cc:511] Master 127.6.138.126:34491 requested a full tablet report, sending...
I20260807 18:42:21.223258 1980 ts_manager.cc:194] Registered new tserver with Master: 15f4668220114143ad73c5726a42be41 (127.6.138.65:40139)
I20260807 18:42:21.224499 1980 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.65:48797
W20260807 18:42:21.342404 2293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:21.342679 2293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:21.342717 2293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:21.347000 2293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:21.347069 2293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:21.347150 2293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:42:21.352065 2293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:21.353407 2293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:21.354892 2293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:21.364777 2310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:21.364825 2304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:21.364821 2306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:21.365075 2293 server_base.cc:1034] running on GCE node
I20260807 18:42:21.365464 2293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:21.366173 2293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:21.367363 2293 hybrid_clock.cc:648] HybridClock initialized: now 1786128141367350 us; error 26 us; skew 500 ppm
I20260807 18:42:21.369302 2293 webserver.cc:460] Webserver started at http://127.6.138.66:44811/ using document root <none> and password file <none>
I20260807 18:42:21.369949 2293 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:21.370051 2293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:21.370293 2293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:21.372537 2293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "2ec433f0f551412a8dbf51fa3e2418ab"
format_stamp: "Formatted at 2026-08-07 18:42:21 on dist-test-slave-2d19"
server_key: "6a766abbba4e654ff1c4c81b34fc9c30"
server_key_iv: "2a458735b58ef3963df2671117358238"
server_key_version: "encryptionkey@0"
I20260807 18:42:21.373118 2293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "2ec433f0f551412a8dbf51fa3e2418ab"
format_stamp: "Formatted at 2026-08-07 18:42:21 on dist-test-slave-2d19"
server_key: "6a766abbba4e654ff1c4c81b34fc9c30"
server_key_iv: "2a458735b58ef3963df2671117358238"
server_key_version: "encryptionkey@0"
I20260807 18:42:21.376812 2293 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:42:21.379221 2319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:21.380611 2293 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:21.380748 2293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "2ec433f0f551412a8dbf51fa3e2418ab"
format_stamp: "Formatted at 2026-08-07 18:42:21 on dist-test-slave-2d19"
server_key: "6a766abbba4e654ff1c4c81b34fc9c30"
server_key_iv: "2a458735b58ef3963df2671117358238"
server_key_version: "encryptionkey@0"
I20260807 18:42:21.380862 2293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:21.399768 2293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:21.400586 2293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:21.400797 2293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:21.401448 2293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:21.402457 2293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:21.402529 2293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:21.402595 2293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:21.402638 2293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:21.412117 2293 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:37743
I20260807 18:42:21.412142 2493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:37743 every 8 connection(s)
I20260807 18:42:21.413137 2293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:42:21.419335 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 2293
I20260807 18:42:21.419503 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:42:21.419829 6697 external_mini_cluster.cc:1428] Setting key 405c409190644f65dbeee2311ed6b61a
I20260807 18:42:21.421873 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34491
--builtin_ntp_servers=127.6.138.84:46253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:42:21.423401 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.415247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:58819 (local address 127.6.138.126:34491)
0807 18:42:21.415398 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.415402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.416654 (+ 1252us) server_negotiation.cc:407] Connection header received
0807 18:42:21.418187 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.418191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.418250 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.418332 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:21.419295 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.420000 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.421248 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.421494 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.421618 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.422838 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:21.422842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:21.422845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:21.422859 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:21.422884 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:21.422894 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:21.422994 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.423169 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.423233 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:42:21.424680 2499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.415685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:34491 (local address 127.6.138.66:58819)
0807 18:42:21.416497 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:21.416538 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:21.417952 (+ 1414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:21.418507 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:21.418514 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:21.418600 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:21.419145 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.419159 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.420143 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.420147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.421114 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.421124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.421661 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.421665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.421835 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.422545 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:42:21.422547 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:21.422590 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:42:21.422657 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:21.423009 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:21.423022 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:21.423035 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 18:42:21.423168 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 18:42:21.423470 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":116,"threads_started":1}
I20260807 18:42:21.425958 2494 heartbeater.cc:347] Connected to a master server at 127.6.138.126:34491
I20260807 18:42:21.426249 2494 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:21.426810 2494 heartbeater.cc:511] Master 127.6.138.126:34491 requested a full tablet report, sending...
I20260807 18:42:21.428000 1980 ts_manager.cc:194] Registered new tserver with Master: 2ec433f0f551412a8dbf51fa3e2418ab (127.6.138.66:37743)
I20260807 18:42:21.428647 1980 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.66:58819
W20260807 18:42:21.541536 2501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:21.541803 2501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:21.541842 2501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:21.545886 2501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:21.545945 2501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:21.546002 2501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:42:21.550957 2501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:34491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:21.552147 2501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:21.553649 2501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:21.563812 2510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:21.563812 2511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:21.563815 2513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:21.564129 2501 server_base.cc:1034] running on GCE node
I20260807 18:42:21.564659 2501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:21.565392 2501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:21.566629 2501 hybrid_clock.cc:648] HybridClock initialized: now 1786128141566607 us; error 37 us; skew 500 ppm
I20260807 18:42:21.568962 2501 webserver.cc:460] Webserver started at http://127.6.138.67:40529/ using document root <none> and password file <none>
I20260807 18:42:21.570158 2501 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:21.570262 2501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:21.570509 2501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:21.572962 2501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "3b5deb69c12447ad8d243a931db82afe"
format_stamp: "Formatted at 2026-08-07 18:42:21 on dist-test-slave-2d19"
server_key: "c1926d2f5e0ed17c9a905f34b87e66de"
server_key_iv: "2147c7e4664e69c30fe9040fcdf82aa0"
server_key_version: "encryptionkey@0"
I20260807 18:42:21.573575 2501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "3b5deb69c12447ad8d243a931db82afe"
format_stamp: "Formatted at 2026-08-07 18:42:21 on dist-test-slave-2d19"
server_key: "c1926d2f5e0ed17c9a905f34b87e66de"
server_key_iv: "2147c7e4664e69c30fe9040fcdf82aa0"
server_key_version: "encryptionkey@0"
I20260807 18:42:21.577464 2501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:42:21.580034 2520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:21.581385 2501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:42:21.581526 2501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "3b5deb69c12447ad8d243a931db82afe"
format_stamp: "Formatted at 2026-08-07 18:42:21 on dist-test-slave-2d19"
server_key: "c1926d2f5e0ed17c9a905f34b87e66de"
server_key_iv: "2147c7e4664e69c30fe9040fcdf82aa0"
server_key_version: "encryptionkey@0"
I20260807 18:42:21.581643 2501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:21.613620 2501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:21.614428 2501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:21.614656 2501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:21.615294 2501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:21.616364 2501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:21.616457 2501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:21.616539 2501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:21.616571 2501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:21.632541 2501 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:43331
I20260807 18:42:21.632586 2684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:43331 every 8 connection(s)
I20260807 18:42:21.633942 2501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:42:21.641937 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 2501
I20260807 18:42:21.642061 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:42:21.642330 6697 external_mini_cluster.cc:1428] Setting key ebb847057424fb56b0ba751e92544cf4
I20260807 18:42:21.648334 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.636430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:49509 (local address 127.6.138.126:34491)
0807 18:42:21.636627 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.636635 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.638095 (+ 1460us) server_negotiation.cc:407] Connection header received
0807 18:42:21.640543 (+ 2448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.640546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.640606 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.640749 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:21.642227 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.643245 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.645502 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.645889 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.645995 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.647619 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:21.647626 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:21.647632 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:21.647648 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:21.647671 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:21.647686 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:21.647752 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.648007 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.648106 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:42:21.649639 2687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.636952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:34491 (local address 127.6.138.67:49509)
0807 18:42:21.637911 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:21.637980 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:21.640192 (+ 2212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:21.640969 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:21.640982 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:21.641121 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:21.641997 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.642025 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.643495 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.643499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.645315 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.645332 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.646080 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.646086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.646346 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.647224 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:42:21.647228 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:21.647300 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:42:21.647413 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:21.647767 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:21.647780 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:21.647789 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:42:21.647968 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 18:42:21.648372 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":150,"threads_started":1}
I20260807 18:42:21.651258 2685 heartbeater.cc:347] Connected to a master server at 127.6.138.126:34491
I20260807 18:42:21.651573 2685 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:21.652218 2685 heartbeater.cc:511] Master 127.6.138.126:34491 requested a full tablet report, sending...
I20260807 18:42:21.653575 1980 ts_manager.cc:194] Registered new tserver with Master: 3b5deb69c12447ad8d243a931db82afe (127.6.138.67:43331)
I20260807 18:42:21.654349 1980 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.67:49509
I20260807 18:42:21.656225 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:42:21.662202 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.658508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.6.138.126:34491)
0807 18:42:21.658659 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.658663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.658853 (+ 190us) server_negotiation.cc:407] Connection header received
0807 18:42:21.658903 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.658905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.658945 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.659023 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:21.659237 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.659720 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.660669 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.660964 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.661066 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.661816 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:21.661819 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:21.661821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:21.661833 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:21.661853 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:21.661861 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:21.661978 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.662041 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.662092 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:42:21.665115 1980 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:42:21.668346 1980 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:42:21.684110 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.677691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.6.138.67:43331)
0807 18:42:21.679063 (+ 1372us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.679072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.679089 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:42:21.679150 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.679154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.679292 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.679455 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:21.679813 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.680847 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.682534 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.683346 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.683451 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.683601 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.683852 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.683954 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1283,"thread_start_us":91,"threads_started":1}
I20260807 18:42:21.684649 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.678371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37942 (local address 127.6.138.66:37743)
0807 18:42:21.678887 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.678896 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.679770 (+ 874us) server_negotiation.cc:407] Connection header received
0807 18:42:21.680002 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.680008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.680176 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.680355 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:21.680566 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.681637 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.682970 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.683718 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.683814 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.683987 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.684348 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.684448 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":130,"threads_started":1}
I20260807 18:42:21.684088 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.677600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:40139 (local address 127.0.0.1:41308)
0807 18:42:21.678802 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:21.678824 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:21.678946 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:21.679420 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:21.679427 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:21.679461 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:21.679627 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.679634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.681019 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.681023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.682253 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.682263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.683640 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.683644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.683747 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.683775 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:42:21.683851 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 18:42:21.683922 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1139,"mutex_wait_us":33,"spinlock_wait_cycles":5888,"thread_start_us":96,"threads_started":1}
I20260807 18:42:21.684918 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.677547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43331 (local address 127.0.0.1:46090)
0807 18:42:21.678400 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:21.678450 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:21.678601 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:21.679430 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:21.679433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:21.679464 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:21.679661 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.679669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.680970 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.680973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.682401 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.682413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.683465 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.683468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.683580 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.683624 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:42:21.683760 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 18:42:21.683888 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":161,"spinlock_wait_cycles":7296,"thread_start_us":96,"threads_started":1}
I20260807 18:42:21.685092 2706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.678371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41308 (local address 127.6.138.65:40139)
0807 18:42:21.678767 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.678774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.678922 (+ 148us) server_negotiation.cc:407] Connection header received
0807 18:42:21.679096 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.679101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.679260 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.679429 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:21.679742 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.680873 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.682374 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.683177 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.683350 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.683578 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.684211 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.684338 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":91,"threads_started":1}
I20260807 18:42:21.685360 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.677890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:37743 (local address 127.0.0.1:37942)
0807 18:42:21.679679 (+ 1789us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:21.679693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:21.679839 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:21.680336 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:21.680339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:21.680358 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:21.680448 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.680453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.681760 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.681764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.682854 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.682863 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.684056 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.684061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.684144 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.684177 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:42:21.685162 (+ 985us) client_negotiation.cc:240] Negotiation successful
0807 18:42:21.685235 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1730,"mutex_wait_us":53,"thread_start_us":94,"threads_started":1}
I20260807 18:42:21.687827 2591 tablet_service.cc:1467] Processing CreateTablet for tablet 66591adeebbc4bb29cb1b435aab827b2 (DEFAULT_TABLE table=test-table [id=dd03d418c4834f9293665ef18e34de8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:21.687952 2184 tablet_service.cc:1467] Processing CreateTablet for tablet 66591adeebbc4bb29cb1b435aab827b2 (DEFAULT_TABLE table=test-table [id=dd03d418c4834f9293665ef18e34de8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:21.689021 2591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66591adeebbc4bb29cb1b435aab827b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:21.689025 2184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66591adeebbc4bb29cb1b435aab827b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:21.695137 2709 tablet_bootstrap.cc:492] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: Bootstrap starting.
I20260807 18:42:21.695082 2395 tablet_service.cc:1467] Processing CreateTablet for tablet 66591adeebbc4bb29cb1b435aab827b2 (DEFAULT_TABLE table=test-table [id=dd03d418c4834f9293665ef18e34de8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:21.696163 2395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66591adeebbc4bb29cb1b435aab827b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:21.696347 2710 tablet_bootstrap.cc:492] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41: Bootstrap starting.
I20260807 18:42:21.697849 2709 tablet_bootstrap.cc:654] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:21.698453 2710 tablet_bootstrap.cc:654] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:21.698798 2709 log.cc:824] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:21.699296 2710 log.cc:824] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:21.701036 2710 tablet_bootstrap.cc:492] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41: No bootstrap required, opened a new log
I20260807 18:42:21.701037 2709 tablet_bootstrap.cc:492] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: No bootstrap required, opened a new log
I20260807 18:42:21.701247 2710 ts_tablet_manager.cc:1397] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:42:21.701248 2709 ts_tablet_manager.cc:1397] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 18:42:21.703227 2713 tablet_bootstrap.cc:492] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab: Bootstrap starting.
I20260807 18:42:21.705132 2709 raft_consensus.cc:348] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } }
I20260807 18:42:21.705147 2710 raft_consensus.cc:348] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } }
I20260807 18:42:21.705385 2710 raft_consensus.cc:374] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:21.705451 2710 raft_consensus.cc:729] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f4668220114143ad73c5726a42be41, State: Initialized, Role: FOLLOWER
I20260807 18:42:21.705566 2713 tablet_bootstrap.cc:654] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:21.705379 2709 raft_consensus.cc:374] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:21.705659 2709 raft_consensus.cc:729] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5deb69c12447ad8d243a931db82afe, State: Initialized, Role: FOLLOWER
I20260807 18:42:21.706067 2710 consensus_queue.cc:260] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } }
I20260807 18:42:21.706089 2709 consensus_queue.cc:260] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } }
I20260807 18:42:21.706439 2713 log.cc:824] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:21.707037 2287 heartbeater.cc:503] Master 127.6.138.126:34491 was elected leader, sending a full tablet report...
I20260807 18:42:21.707177 2709 ts_tablet_manager.cc:1428] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:42:21.707554 2685 heartbeater.cc:503] Master 127.6.138.126:34491 was elected leader, sending a full tablet report...
I20260807 18:42:21.708333 2710 ts_tablet_manager.cc:1428] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 18:42:21.708674 2713 tablet_bootstrap.cc:492] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab: No bootstrap required, opened a new log
I20260807 18:42:21.708889 2713 ts_tablet_manager.cc:1397] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
W20260807 18:42:21.710261 2290 tablet.cc:2369] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:42:21.712908 2713 raft_consensus.cc:348] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } }
I20260807 18:42:21.713151 2713 raft_consensus.cc:374] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:21.713225 2713 raft_consensus.cc:729] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec433f0f551412a8dbf51fa3e2418ab, State: Initialized, Role: FOLLOWER
I20260807 18:42:21.713621 2713 consensus_queue.cc:260] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } }
I20260807 18:42:21.714259 2494 heartbeater.cc:503] Master 127.6.138.126:34491 was elected leader, sending a full tablet report...
I20260807 18:42:21.714627 2713 ts_tablet_manager.cc:1428] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260807 18:42:21.885155 2686 tablet.cc:2369] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:42:21.902563 2717 raft_consensus.cc:482] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:21.902784 2717 raft_consensus.cc:504] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } }
I20260807 18:42:21.903990 2717 leader_election.cc:288] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b5deb69c12447ad8d243a931db82afe (127.6.138.67:43331), 15f4668220114143ad73c5726a42be41 (127.6.138.65:40139)
I20260807 18:42:21.908149 2706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.904436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:51505 (local address 127.6.138.65:40139)
0807 18:42:21.904575 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.904579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.904595 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:42:21.904761 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.904764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.904809 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.904899 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:21.905234 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.905739 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.907018 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.907629 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.907723 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.907763 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.907977 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.908045 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:42:21.908165 2499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.904320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:40139 (local address 127.6.138.66:51505)
0807 18:42:21.904463 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:21.904490 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:21.904609 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:21.904915 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:21.904918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:21.904962 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:21.905085 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.905097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.905846 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.905850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.906881 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.906890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.907742 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.907746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.907847 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.907872 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:42:21.907954 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 18:42:21.908024 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"spinlock_wait_cycles":1408}
I20260807 18:42:21.908910 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.904320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:43331 (local address 127.6.138.66:42727)
0807 18:42:21.904802 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:21.904814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:21.905026 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:21.905348 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:21.905351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:21.905367 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:21.905498 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.905504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.906463 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.906465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.907446 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:21.907457 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.908437 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.908441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:21.908571 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.908593 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:42:21.908676 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 18:42:21.908750 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":145,"spinlock_wait_cycles":126592,"thread_start_us":141,"threads_started":1}
I20260807 18:42:21.908927 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.904443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:42727 (local address 127.6.138.67:43331)
0807 18:42:21.904578 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.904583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.904894 (+ 311us) server_negotiation.cc:407] Connection header received
0807 18:42:21.905169 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.905175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.905236 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.905324 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:21.905626 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.906340 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.907578 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.908171 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.908322 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.908379 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.908743 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.908820 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:42:21.909016 2217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66591adeebbc4bb29cb1b435aab827b2" candidate_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f4668220114143ad73c5726a42be41" is_pre_election: true
I20260807 18:42:21.909394 2217 raft_consensus.cc:2454] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec433f0f551412a8dbf51fa3e2418ab in term 0.
I20260807 18:42:21.909584 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66591adeebbc4bb29cb1b435aab827b2" candidate_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b5deb69c12447ad8d243a931db82afe" is_pre_election: true
I20260807 18:42:21.909895 2616 raft_consensus.cc:2454] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec433f0f551412a8dbf51fa3e2418ab in term 0.
I20260807 18:42:21.909965 2326 leader_election.cc:302] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f4668220114143ad73c5726a42be41, 2ec433f0f551412a8dbf51fa3e2418ab; no voters:
I20260807 18:42:21.910261 2717 raft_consensus.cc:2781] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:42:21.910380 2717 raft_consensus.cc:482] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:42:21.910437 2717 raft_consensus.cc:3037] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:21.911484 2717 raft_consensus.cc:504] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } }
I20260807 18:42:21.911862 2717 leader_election.cc:288] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [CANDIDATE]: Term 1 election: Requested vote from peers 3b5deb69c12447ad8d243a931db82afe (127.6.138.67:43331), 15f4668220114143ad73c5726a42be41 (127.6.138.65:40139)
I20260807 18:42:21.912441 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66591adeebbc4bb29cb1b435aab827b2" candidate_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b5deb69c12447ad8d243a931db82afe"
I20260807 18:42:21.912441 2217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66591adeebbc4bb29cb1b435aab827b2" candidate_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f4668220114143ad73c5726a42be41"
I20260807 18:42:21.912595 2217 raft_consensus.cc:3037] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:21.912595 2616 raft_consensus.cc:3037] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:21.914064 2616 raft_consensus.cc:2454] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec433f0f551412a8dbf51fa3e2418ab in term 1.
I20260807 18:42:21.914064 2217 raft_consensus.cc:2454] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec433f0f551412a8dbf51fa3e2418ab in term 1.
W20260807 18:42:21.914319 2495 tablet.cc:2369] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:42:21.914417 2324 leader_election.cc:302] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec433f0f551412a8dbf51fa3e2418ab, 3b5deb69c12447ad8d243a931db82afe; no voters:
I20260807 18:42:21.914606 2717 raft_consensus.cc:2781] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:21.914873 2717 raft_consensus.cc:686] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 1 LEADER]: Becoming Leader. State: Replica: 2ec433f0f551412a8dbf51fa3e2418ab, State: Running, Role: LEADER
I20260807 18:42:21.915275 2717 consensus_queue.cc:237] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } }
I20260807 18:42:21.918919 1979 catalog_manager.cc:5495] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ec433f0f551412a8dbf51fa3e2418ab (127.6.138.66). New cstate: current_term: 1 leader_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec433f0f551412a8dbf51fa3e2418ab" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 37743 } health_report { overall_health: HEALTHY } } }
I20260807 18:42:21.976404 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:21.972679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37954 (local address 127.6.138.66:37743)
0807 18:42:21.972831 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:21.972836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:21.972853 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:42:21.972953 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:21.972956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:21.973043 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:21.973168 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:42:21.973423 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.973925 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.974811 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:21.975038 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:21.975170 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:21.975536 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:42:21.975953 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:42:21.976104 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:21.976170 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 18:42:21.976226 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:42:21.982348 2217 raft_consensus.cc:1261] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41 [term 1 FOLLOWER]: Refusing update from remote peer 2ec433f0f551412a8dbf51fa3e2418ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:21.982362 2616 raft_consensus.cc:1261] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [term 1 FOLLOWER]: Refusing update from remote peer 2ec433f0f551412a8dbf51fa3e2418ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:21.983103 2719 consensus_queue.cc:1035] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f4668220114143ad73c5726a42be41" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:21.983345 2717 consensus_queue.cc:1035] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b5deb69c12447ad8d243a931db82afe" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 43331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:21.992237 2728 mvcc.cc:205] Tried to move back new op lower bound from 7315980869552541696 to 7315980869288472576. Current Snapshot: MvccSnapshot[applied={T|T < 7315980869552541696}]
I20260807 18:42:21.999723 1980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:21.999935 1980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:22.002327 1980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a92d62c8a474fa9bf8994f4d006c308: Sending DeleteTablet for 3 replicas of tablet 66591adeebbc4bb29cb1b435aab827b2
I20260807 18:42:22.003118 2184 tablet_service.cc:1514] Processing DeleteTablet for tablet 66591adeebbc4bb29cb1b435aab827b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:21 UTC) from {username='slave'} at 127.0.0.1:41308
I20260807 18:42:22.003141 2591 tablet_service.cc:1514] Processing DeleteTablet for tablet 66591adeebbc4bb29cb1b435aab827b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:21 UTC) from {username='slave'} at 127.0.0.1:46090
I20260807 18:42:22.003283 2395 tablet_service.cc:1514] Processing DeleteTablet for tablet 66591adeebbc4bb29cb1b435aab827b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:21 UTC) from {username='slave'} at 127.0.0.1:37942
I20260807 18:42:22.003713 2741 tablet_replica.cc:330] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab: stopping tablet replica
I20260807 18:42:22.003906 2739 tablet_replica.cc:330] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41: stopping tablet replica
I20260807 18:42:22.003953 2740 tablet_replica.cc:330] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: stopping tablet replica
I20260807 18:42:22.004297 2740 raft_consensus.cc:2229] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:42:22.004299 2739 raft_consensus.cc:2229] T 66591adeebbc4bb29cb1b435aab827b2 P 15f4668220114143ad73c5726a42be41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:42:22.004451 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 2086
I20260807 18:42:22.007162 2740 raft_consensus.cc:2258] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:22.015435 2740 ts_tablet_manager.cc:1905] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:22.017484 2741 raft_consensus.cc:2229] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:42:22.017999 2741 raft_consensus.cc:2258] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:22.019310 2740 ts_tablet_manager.cc:1918] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:22.019425 2740 log.cc:1196] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/66591adeebbc4bb29cb1b435aab827b2
I20260807 18:42:22.019706 2741 ts_tablet_manager.cc:1905] T 66591adeebbc4bb29cb1b435aab827b2 P 2ec433f0f551412a8dbf51fa3e2418ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:22.019780 2740 ts_tablet_manager.cc:1939] T 66591adeebbc4bb29cb1b435aab827b2 P 3b5deb69c12447ad8d243a931db82afe: Deleting consensus metadata
W20260807 18:42:22.021159 1951 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:40139 (error 108)
I20260807 18:42:22.021265 1949 catalog_manager.cc:4841] TS 3b5deb69c12447ad8d243a931db82afe (127.6.138.67:43331): tablet 66591adeebbc4bb29cb1b435aab827b2 (table test-table [id=dd03d418c4834f9293665ef18e34de8a]) successfully deleted
I20260807 18:42:22.022292 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:22.021962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:40139 (local address 127.0.0.1:41310)
0807 18:42:22.022107 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:22.022187 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:42:22.022455 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 2293
W20260807 18:42:22.028733 1951 catalog_manager.cc:4584] TS 15f4668220114143ad73c5726a42be41 (127.6.138.65:40139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 66591adeebbc4bb29cb1b435aab827b2: Network error: Client connection negotiation failed: client connection to 127.6.138.65:40139: connect: Connection refused (error 111)
I20260807 18:42:22.035949 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 2501
I20260807 18:42:22.036644 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:22.036139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:37743 (local address 127.0.0.1:37964)
0807 18:42:22.036453 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:22.036532 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.66:37743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 18:42:22.048600 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 1923
2026-08-07T18:42:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1344 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:22 dist-test-slave-2d19 krb5kdc[2747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:22 dist-test-slave-2d19 krb5kdc[2747](info): set up 2 sockets
Aug 07 18:42:22 dist-test-slave-2d19 krb5kdc[2747](info): commencing operation
krb5kdc: starting...
W20260807 18:42:23.407096 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:23 dist-test-slave-2d19 krb5kdc[2747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:42:23.544016 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42237
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:23.657804 2762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:23.658120 2762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:23.658198 2762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:23.662175 2762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:23.662271 2762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:23.662309 2762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:23.662364 2762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:23.662400 2762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:23.667625 2762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:23.668743 2762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:23.669806 2762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:23.678622 2769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:23.678615 2777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:23.678615 2771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:23.678769 2762 server_base.cc:1034] running on GCE node
I20260807 18:42:23.679618 2762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:23.680848 2762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:23.682050 2762 hybrid_clock.cc:648] HybridClock initialized: now 1786128143682041 us; error 32 us; skew 500 ppm
Aug 07 18:42:23 dist-test-slave-2d19 krb5kdc[2747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:23.691190 2762 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:23.692121 2762 webserver.cc:460] Webserver started at http://127.6.138.126:34389/ using document root <none> and password file <none>
I20260807 18:42:23.692736 2762 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:23.692800 2762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:23.692966 2762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:23.695034 2762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "6220dd087da64a57ae241511b2b563d9"
format_stamp: "Formatted at 2026-08-07 18:42:23 on dist-test-slave-2d19"
server_key: "487322fd17b5b9479a20cb2c51109ea0"
server_key_iv: "358cbcbef176516cc69edfe666a545b0"
server_key_version: "encryptionkey@0"
I20260807 18:42:23.695575 2762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "6220dd087da64a57ae241511b2b563d9"
format_stamp: "Formatted at 2026-08-07 18:42:23 on dist-test-slave-2d19"
server_key: "487322fd17b5b9479a20cb2c51109ea0"
server_key_iv: "358cbcbef176516cc69edfe666a545b0"
server_key_version: "encryptionkey@0"
I20260807 18:42:23.699285 2762 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 18:42:23.701748 2787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:23.702996 2762 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 18:42:23.703097 2762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "6220dd087da64a57ae241511b2b563d9"
format_stamp: "Formatted at 2026-08-07 18:42:23 on dist-test-slave-2d19"
server_key: "487322fd17b5b9479a20cb2c51109ea0"
server_key_iv: "358cbcbef176516cc69edfe666a545b0"
server_key_version: "encryptionkey@0"
I20260807 18:42:23.703186 2762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:23.718097 2762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:23.720968 2762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:23.721179 2762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:23.728051 2762 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:42237
I20260807 18:42:23.728117 2861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:42237 every 8 connection(s)
I20260807 18:42:23.729070 2762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:23.731387 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 2762
I20260807 18:42:23.731549 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:23.731878 6697 external_mini_cluster.cc:1428] Setting key 625908d73d9f936db00ae1067b3ab48a
I20260807 18:42:23.732306 2862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:23.740105 2862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9: Bootstrap starting.
I20260807 18:42:23.743067 2862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9: Neither blocks nor log segments found. Creating new log.
Aug 07 18:42:23 dist-test-slave-2d19 krb5kdc[2747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:23.743988 2862 log.cc:824] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:23.746227 2862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9: No bootstrap required, opened a new log
I20260807 18:42:23.748819 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:23.733437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46134 (local address 127.6.138.126:42237)
0807 18:42:23.734131 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:23.734155 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:23.734221 (+ 66us) server_negotiation.cc:407] Connection header received
0807 18:42:23.736271 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:23.736298 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:23.736737 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:23.737027 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:23.737995 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:23.738990 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:23.740158 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:23.740752 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:23.740878 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:23.743726 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:23.743733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:23.743753 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:23.743789 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:23.745743 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:23.746249 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:23.746254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:23.746258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:23.746435 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:23.746780 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:23.746782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:23.746783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:23.747200 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:23.747347 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:23.747687 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 18:42:23.748000 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":209,"threads_started":1}
I20260807 18:42:23.750322 2862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6220dd087da64a57ae241511b2b563d9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42237 } }
I20260807 18:42:23.750583 2862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:23.750674 2862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6220dd087da64a57ae241511b2b563d9, State: Initialized, Role: FOLLOWER
I20260807 18:42:23.751175 2862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6220dd087da64a57ae241511b2b563d9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42237 } }
I20260807 18:42:23.751339 2862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:23.751413 2862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:23.751515 2862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:23.752596 2862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6220dd087da64a57ae241511b2b563d9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42237 } }
I20260807 18:42:23.752938 2862 leader_election.cc:302] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6220dd087da64a57ae241511b2b563d9; no voters:
I20260807 18:42:23.753295 2862 leader_election.cc:288] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:23.753428 2869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:23.753696 2869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [term 1 LEADER]: Becoming Leader. State: Replica: 6220dd087da64a57ae241511b2b563d9, State: Running, Role: LEADER
I20260807 18:42:23.754062 2869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6220dd087da64a57ae241511b2b563d9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42237 } }
I20260807 18:42:23.754550 2862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:23.755908 2870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6220dd087da64a57ae241511b2b563d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6220dd087da64a57ae241511b2b563d9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42237 } } }
I20260807 18:42:23.755926 2871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6220dd087da64a57ae241511b2b563d9. Latest consensus state: current_term: 1 leader_uuid: "6220dd087da64a57ae241511b2b563d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6220dd087da64a57ae241511b2b563d9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42237 } } }
I20260807 18:42:23.756059 2870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:23.756063 2871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:23.757057 2878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:23.759701 2878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:23.765368 2878 catalog_manager.cc:1347] Generated new cluster ID: 91bead6327584a4cb648c67c38433133
I20260807 18:42:23.765440 2878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:23.775554 2878 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:23.776569 2878 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:23.781513 2878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9: Generated new TSK 0
I20260807 18:42:23.782267 2878 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:42:23.823483 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42237
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:23.954874 2903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:23.955240 2903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:23.955319 2903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:23.959506 2903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:23.959563 2903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:23.959672 2903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:42:23.965035 2903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:23.966068 2903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:23.967087 2903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:42:23.977138 2903 server_base.cc:1034] running on GCE node
W20260807 18:42:23.977164 2909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:23.977164 2914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:23.977469 2910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:23.977916 2903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:23.978498 2903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:23.979657 2903 hybrid_clock.cc:648] HybridClock initialized: now 1786128143979595 us; error 80 us; skew 500 ppm
Aug 07 18:42:23 dist-test-slave-2d19 krb5kdc[2747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:23.982523 2903 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:42:23.983348 2903 webserver.cc:460] Webserver started at http://127.6.138.65:33315/ using document root <none> and password file <none>
I20260807 18:42:23.983856 2903 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:23.983912 2903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:23.984086 2903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:23.986148 2903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6"
format_stamp: "Formatted at 2026-08-07 18:42:23 on dist-test-slave-2d19"
server_key: "de63aecbecdb8bdb9f23e9425318993c"
server_key_iv: "58b6fe6a7af4e9fdf324976e19843dee"
server_key_version: "encryptionkey@0"
I20260807 18:42:23.986646 2903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6"
format_stamp: "Formatted at 2026-08-07 18:42:23 on dist-test-slave-2d19"
server_key: "de63aecbecdb8bdb9f23e9425318993c"
server_key_iv: "58b6fe6a7af4e9fdf324976e19843dee"
server_key_version: "encryptionkey@0"
I20260807 18:42:23.990357 2903 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 18:42:23.993000 2922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:23.994257 2903 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:23.994406 2903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6"
format_stamp: "Formatted at 2026-08-07 18:42:23 on dist-test-slave-2d19"
server_key: "de63aecbecdb8bdb9f23e9425318993c"
server_key_iv: "58b6fe6a7af4e9fdf324976e19843dee"
server_key_version: "encryptionkey@0"
I20260807 18:42:23.994527 2903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:24.019503 2903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:24.022779 2903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:24.022992 2903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:24.023644 2903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:24.024693 2903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:24.024755 2903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:24.024803 2903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:24.024822 2903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:24.033926 2903 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:37525
I20260807 18:42:24.033954 3091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:37525 every 8 connection(s)
I20260807 18:42:24.034837 2903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:42:24.042443 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 2903
I20260807 18:42:24.042589 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:42:24.042950 6697 external_mini_cluster.cc:1428] Setting key f44984e1c6f1a1f1b509c3687932b316
Aug 07 18:42:24 dist-test-slave-2d19 krb5kdc[2747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:24.049666 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.036795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:57759 (local address 127.6.138.126:42237)
0807 18:42:24.036937 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.036941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.037814 (+ 873us) server_negotiation.cc:407] Connection header received
0807 18:42:24.039140 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.039143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.039191 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.039269 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:24.040526 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.041103 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.042283 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.042665 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.042789 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.045408 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:24.045415 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:24.045421 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:24.045458 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:24.047253 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:24.047922 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:24.047925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:24.047926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:24.048038 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:24.048488 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:24.048491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:24.048492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:24.048671 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:24.048748 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.049426 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.049540 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:42:24.050961 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.037177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42237 (local address 127.6.138.65:57759)
0807 18:42:24.037637 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:24.037683 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:24.038921 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:24.039438 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:24.039445 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:24.039816 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:24.040365 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.040382 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.041232 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.041235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.042148 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.042158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.042819 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.042822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.042960 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.043638 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:24.043665 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:24.045229 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:24.047386 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:24.047396 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:24.047414 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:24.047810 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:24.048133 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:24.048137 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:24.048142 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:24.048378 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:24.048783 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:24.048793 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:24.049149 (+ 356us) client_negotiation.cc:769] Sending connection context
0807 18:42:24.049368 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 18:42:24.049746 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":117,"threads_started":1}
I20260807 18:42:24.052469 3092 heartbeater.cc:347] Connected to a master server at 127.6.138.126:42237
I20260807 18:42:24.052738 3092 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:24.053249 3092 heartbeater.cc:511] Master 127.6.138.126:42237 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:42:24.054921 2807 ts_manager.cc:194] Registered new tserver with Master: 9e4b76e1d15f4af9b0ffcb7ac4d413b6 (127.6.138.65:37525)
I20260807 18:42:24.056368 2807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:57759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:42:24.076414 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42237
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:24.192209 3102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:24.192572 3102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:24.192656 3102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:24.196904 3102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:24.196990 3102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:24.197079 3102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:42:24.202668 3102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:24.203714 3102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:24.204825 3102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:24.214951 3109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:24.214951 3111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:24.214951 3115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:24.215268 3102 server_base.cc:1034] running on GCE node
I20260807 18:42:24.215664 3102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:24.216409 3102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:24.217578 3102 hybrid_clock.cc:648] HybridClock initialized: now 1786128144217559 us; error 30 us; skew 500 ppm
Aug 07 18:42:24 dist-test-slave-2d19 krb5kdc[2747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128144, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:24.220548 3102 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:42:24.221421 3102 webserver.cc:460] Webserver started at http://127.6.138.66:45127/ using document root <none> and password file <none>
I20260807 18:42:24.221968 3102 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:24.222026 3102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:24.222191 3102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:24.224226 3102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "5dbafa8247224647bee74704bb35f5c8"
format_stamp: "Formatted at 2026-08-07 18:42:24 on dist-test-slave-2d19"
server_key: "23a23dc48d3fcbacd46dfb537f446b4c"
server_key_iv: "1c2c07f0131432cd8644da66ca3b89de"
server_key_version: "encryptionkey@0"
I20260807 18:42:24.224838 3102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "5dbafa8247224647bee74704bb35f5c8"
format_stamp: "Formatted at 2026-08-07 18:42:24 on dist-test-slave-2d19"
server_key: "23a23dc48d3fcbacd46dfb537f446b4c"
server_key_iv: "1c2c07f0131432cd8644da66ca3b89de"
server_key_version: "encryptionkey@0"
I20260807 18:42:24.228667 3102 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:42:24.231218 3124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:24.232725 3102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:42:24.232872 3102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "5dbafa8247224647bee74704bb35f5c8"
format_stamp: "Formatted at 2026-08-07 18:42:24 on dist-test-slave-2d19"
server_key: "23a23dc48d3fcbacd46dfb537f446b4c"
server_key_iv: "1c2c07f0131432cd8644da66ca3b89de"
server_key_version: "encryptionkey@0"
I20260807 18:42:24.232988 3102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:24.284229 3102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:24.287947 3102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:24.288168 3102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:24.288828 3102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:24.289696 3102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:24.289769 3102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:24.289844 3102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:24.289896 3102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:24.299665 3102 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:36647
I20260807 18:42:24.299698 3312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:36647 every 8 connection(s)
I20260807 18:42:24.300632 3102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:42:24.304611 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 3102
I20260807 18:42:24.304782 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:42:24.305181 6697 external_mini_cluster.cc:1428] Setting key 098817eea715e186fe47d179556e4166
Aug 07 18:42:24 dist-test-slave-2d19 krb5kdc[2747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128144, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:42:24.317588 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.302524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:49923 (local address 127.6.138.126:42237)
0807 18:42:24.302668 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.302672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.303621 (+ 949us) server_negotiation.cc:407] Connection header received
0807 18:42:24.305139 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.305143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.305238 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.305351 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:24.307244 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.307835 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.309756 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.310024 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.310158 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.313204 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:24.313211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:24.313218 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:24.313255 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:24.315058 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:24.315634 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:24.315638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:24.315643 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:24.315739 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:24.316232 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:24.316239 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:24.316241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:24.316533 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:24.316651 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.317238 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.317385 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:42:24.318718 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.302969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42237 (local address 127.6.138.66:49923)
0807 18:42:24.303450 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:24.303495 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:24.304877 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:24.305715 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:24.305723 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:24.306326 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:24.307081 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.307105 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.307971 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.307976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.309607 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.309619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.310205 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.310209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.310410 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.311077 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:24.311095 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:24.313002 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:24.315190 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:24.315196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:24.315210 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:24.315501 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:24.315878 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:24.315881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:24.315929 (+ 48us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:24.316096 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:24.316676 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:24.316682 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:24.316941 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 18:42:24.317165 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 18:42:24.317539 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":130,"threads_started":1}
I20260807 18:42:24.320015 3314 heartbeater.cc:347] Connected to a master server at 127.6.138.126:42237
I20260807 18:42:24.320313 3314 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
I20260807 18:42:24.320880 3314 heartbeater.cc:511] Master 127.6.138.126:42237 requested a full tablet report, sending...
I20260807 18:42:24.321971 2807 ts_manager.cc:194] Registered new tserver with Master: 5dbafa8247224647bee74704bb35f5c8 (127.6.138.66:36647)
I20260807 18:42:24.322553 2807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:49923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:42:24.339370 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42237
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:24.458396 3321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:24.458670 3321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:24.458715 3321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:24.462719 3321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:24.462778 3321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:24.462837 3321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:42:24.468194 3321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:42237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:24.469249 3321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:24.470223 3321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:24.479913 3329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:24.479928 3330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:24.480139 3321 server_base.cc:1034] running on GCE node
W20260807 18:42:24.480243 3332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:24.480634 3321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:24.481302 3321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:24.482498 3321 hybrid_clock.cc:648] HybridClock initialized: now 1786128144482484 us; error 26 us; skew 500 ppm
Aug 07 18:42:24 dist-test-slave-2d19 krb5kdc[2747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128144, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:24.485435 3321 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:42:24.486413 3321 webserver.cc:460] Webserver started at http://127.6.138.67:42823/ using document root <none> and password file <none>
I20260807 18:42:24.487017 3321 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:24.487118 3321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:24.487360 3321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:24.489493 3321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "aba2e468ca5045ad8c5dfc2f4522665e"
format_stamp: "Formatted at 2026-08-07 18:42:24 on dist-test-slave-2d19"
server_key: "f8bbbffcfe80c117c4b61607199c5d43"
server_key_iv: "a4d7c4bb7de7837c95b703111aafb0a9"
server_key_version: "encryptionkey@0"
I20260807 18:42:24.490077 3321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "aba2e468ca5045ad8c5dfc2f4522665e"
format_stamp: "Formatted at 2026-08-07 18:42:24 on dist-test-slave-2d19"
server_key: "f8bbbffcfe80c117c4b61607199c5d43"
server_key_iv: "a4d7c4bb7de7837c95b703111aafb0a9"
server_key_version: "encryptionkey@0"
I20260807 18:42:24.494117 3321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:42:24.496667 3340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:24.497947 3321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:42:24.498103 3321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "aba2e468ca5045ad8c5dfc2f4522665e"
format_stamp: "Formatted at 2026-08-07 18:42:24 on dist-test-slave-2d19"
server_key: "f8bbbffcfe80c117c4b61607199c5d43"
server_key_iv: "a4d7c4bb7de7837c95b703111aafb0a9"
server_key_version: "encryptionkey@0"
I20260807 18:42:24.498246 3321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:24.516388 3321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:24.519236 3321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:24.519447 3321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:24.520069 3321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:24.520922 3321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:24.521006 3321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:24.521080 3321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:24.521133 3321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:24.530058 3321 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:45633
I20260807 18:42:24.530087 3498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:45633 every 8 connection(s)
I20260807 18:42:24.530936 3321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:42:24.537220 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 3321
I20260807 18:42:24.537360 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:42:24.537683 6697 external_mini_cluster.cc:1428] Setting key d29195d6d4aaeb3dee9c3c2d33b67769
Aug 07 18:42:24 dist-test-slave-2d19 krb5kdc[2747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128144, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:24.545274 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.532904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:50725 (local address 127.6.138.126:42237)
0807 18:42:24.533061 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.533065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.533950 (+ 885us) server_negotiation.cc:407] Connection header received
0807 18:42:24.535378 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.535381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.535447 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.535560 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:24.536875 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.537730 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.538890 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.539124 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.539244 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.541867 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:24.541877 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:24.541880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:24.541903 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:24.543162 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:24.543687 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:24.543691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:24.543693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:24.543776 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:24.544132 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:24.544137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:24.544138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:24.544380 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:24.544489 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.544979 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.545113 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:42:24.546077 3501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.533257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42237 (local address 127.6.138.67:50725)
0807 18:42:24.533734 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:24.533778 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:24.535153 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:24.535741 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:24.535748 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:24.536142 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:24.536716 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.536730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.537859 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.537862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.538752 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.538762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.539286 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.539290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.539461 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.540061 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:24.540077 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:24.541700 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:24.543291 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:24.543296 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:24.543310 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:24.543578 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:24.543875 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:24.543878 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:24.543880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:24.544032 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:24.544489 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:24.544497 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:24.544742 (+ 245us) client_negotiation.cc:769] Sending connection context
0807 18:42:24.544936 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:42:24.545270 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":114,"threads_started":1}
I20260807 18:42:24.547183 3499 heartbeater.cc:347] Connected to a master server at 127.6.138.126:42237
I20260807 18:42:24.547436 3499 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:24.547940 3499 heartbeater.cc:511] Master 127.6.138.126:42237 requested a full tablet report, sending...
I20260807 18:42:24.549109 2807 ts_manager.cc:194] Registered new tserver with Master: aba2e468ca5045ad8c5dfc2f4522665e (127.6.138.67:45633)
I20260807 18:42:24.549674 2807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:50725
I20260807 18:42:24.551570 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:42:24.561826 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.553741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.6.138.126:42237)
0807 18:42:24.553886 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.553891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.554113 (+ 222us) server_negotiation.cc:407] Connection header received
0807 18:42:24.554204 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.554206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.554245 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.554353 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:24.555096 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.555697 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.556768 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.557018 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.557100 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.558600 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:24.558603 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:24.558605 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:24.558628 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:24.559898 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:24.560472 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:24.560475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:24.560476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:24.560551 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:24.560906 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:24.560908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:24.560909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:24.561121 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:24.561236 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.561527 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.561642 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:42:24.565080 2807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:42:24.568143 2807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:42:24.582995 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.577442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.6.138.65:37525)
0807 18:42:24.577838 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.577845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.578190 (+ 345us) server_negotiation.cc:407] Connection header received
0807 18:42:24.578301 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.578308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.578504 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.578662 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:24.578879 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.579732 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.581726 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.582321 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.582448 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.582611 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.582749 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.582845 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":90,"threads_started":1}
I20260807 18:42:24.582994 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.576729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37525 (local address 127.0.0.1:55518)
0807 18:42:24.577984 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:24.578001 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:24.578110 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:24.578647 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:24.578649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:24.578665 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:24.578761 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.578768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.579854 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.579857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.581568 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.581579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.582448 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.582452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.582558 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.582594 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:42:24.582713 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 18:42:24.582837 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1185,"mutex_wait_us":64,"thread_start_us":86,"threads_started":1}
I20260807 18:42:24.583235 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.576610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:45633 (local address 127.0.0.1:57404)
0807 18:42:24.577510 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:24.577561 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:24.577700 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:24.578307 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:24.578316 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:24.578340 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:24.578470 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.578475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.580119 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.580123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.581568 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.581579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.582783 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.582786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.582892 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.582916 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:42:24.582970 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 18:42:24.583054 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":92,"thread_start_us":96,"threads_started":1}
I20260807 18:42:24.583568 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.577313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.6.138.66:36647)
0807 18:42:24.577993 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.578015 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.578265 (+ 250us) server_negotiation.cc:407] Connection header received
0807 18:42:24.578362 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.578367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.578518 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.578662 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:24.579134 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.580234 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.581528 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.582521 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.582640 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.582796 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.583250 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.583430 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":113,"threads_started":1}
I20260807 18:42:24.584100 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.577011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:36647 (local address 127.0.0.1:36476)
0807 18:42:24.578146 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:24.578159 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:24.578260 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:24.578857 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:24.578861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:24.578880 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:24.578999 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.579006 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.580420 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.580423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.581391 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.581401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.582850 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.582865 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.582973 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.582991 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:42:24.583051 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 18:42:24.583964 (+ 913us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260807 18:42:24.584730 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.576741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.6.138.67:45633)
0807 18:42:24.577779 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.577793 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.577819 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:42:24.577901 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.577906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.578127 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.578638 (+ 511us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:24.578810 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.579976 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.581695 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.582641 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.584048 (+ 1407us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.584501 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.584546 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.584619 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":148,"threads_started":1}
I20260807 18:42:24.586310 2987 tablet_service.cc:1467] Processing CreateTablet for tablet 309b300f0b264a8f99ce0655216fbd1a (DEFAULT_TABLE table=test-table [id=78a316da22634bf9b85a98862d70c5c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:24.586736 3212 tablet_service.cc:1467] Processing CreateTablet for tablet 309b300f0b264a8f99ce0655216fbd1a (DEFAULT_TABLE table=test-table [id=78a316da22634bf9b85a98862d70c5c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:24.587517 2987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309b300f0b264a8f99ce0655216fbd1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:24.587764 3212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309b300f0b264a8f99ce0655216fbd1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:24.587461 3409 tablet_service.cc:1467] Processing CreateTablet for tablet 309b300f0b264a8f99ce0655216fbd1a (DEFAULT_TABLE table=test-table [id=78a316da22634bf9b85a98862d70c5c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:24.588464 3409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309b300f0b264a8f99ce0655216fbd1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:24.594148 3523 tablet_bootstrap.cc:492] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: Bootstrap starting.
I20260807 18:42:24.595732 3524 tablet_bootstrap.cc:492] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6: Bootstrap starting.
I20260807 18:42:24.596627 3525 tablet_bootstrap.cc:492] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: Bootstrap starting.
I20260807 18:42:24.597064 3523 tablet_bootstrap.cc:654] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:24.597719 3524 tablet_bootstrap.cc:654] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:24.598050 3523 log.cc:824] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:24.598553 3524 log.cc:824] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:24.598763 3525 tablet_bootstrap.cc:654] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:24.600046 3525 log.cc:824] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:24.600584 3524 tablet_bootstrap.cc:492] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6: No bootstrap required, opened a new log
I20260807 18:42:24.600648 3523 tablet_bootstrap.cc:492] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: No bootstrap required, opened a new log
I20260807 18:42:24.600780 3524 ts_tablet_manager.cc:1397] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:42:24.600782 3523 ts_tablet_manager.cc:1397] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 18:42:24.602062 3525 tablet_bootstrap.cc:492] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: No bootstrap required, opened a new log
I20260807 18:42:24.602231 3525 ts_tablet_manager.cc:1397] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:42:24.604525 3524 raft_consensus.cc:348] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } }
I20260807 18:42:24.604727 3524 raft_consensus.cc:374] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:24.604765 3524 raft_consensus.cc:729] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4b76e1d15f4af9b0ffcb7ac4d413b6, State: Initialized, Role: FOLLOWER
I20260807 18:42:24.604697 3523 raft_consensus.cc:348] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } }
I20260807 18:42:24.605137 3523 raft_consensus.cc:374] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:24.605204 3523 raft_consensus.cc:729] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dbafa8247224647bee74704bb35f5c8, State: Initialized, Role: FOLLOWER
I20260807 18:42:24.605176 3524 consensus_queue.cc:260] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } }
I20260807 18:42:24.605684 3523 consensus_queue.cc:260] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } }
I20260807 18:42:24.606015 3525 raft_consensus.cc:348] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } }
I20260807 18:42:24.606302 3524 ts_tablet_manager.cc:1428] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:42:24.606324 3525 raft_consensus.cc:374] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:24.606393 3525 raft_consensus.cc:729] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aba2e468ca5045ad8c5dfc2f4522665e, State: Initialized, Role: FOLLOWER
I20260807 18:42:24.606688 3523 ts_tablet_manager.cc:1428] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:42:24.606757 3092 heartbeater.cc:503] Master 127.6.138.126:42237 was elected leader, sending a full tablet report...
I20260807 18:42:24.606834 3525 consensus_queue.cc:260] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } }
I20260807 18:42:24.607039 3314 heartbeater.cc:503] Master 127.6.138.126:42237 was elected leader, sending a full tablet report...
I20260807 18:42:24.607844 3499 heartbeater.cc:503] Master 127.6.138.126:42237 was elected leader, sending a full tablet report...
I20260807 18:42:24.607904 3525 ts_tablet_manager.cc:1428] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:42:24.615166 3531 raft_consensus.cc:482] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:24.615319 3531 raft_consensus.cc:504] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } }
I20260807 18:42:24.616405 3531 leader_election.cc:288] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aba2e468ca5045ad8c5dfc2f4522665e (127.6.138.67:45633), 9e4b76e1d15f4af9b0ffcb7ac4d413b6 (127.6.138.65:37525)
I20260807 18:42:24.621594 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.616618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:47839 (local address 127.6.138.67:45633)
0807 18:42:24.616959 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.616962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.616974 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:42:24.617045 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.617049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.617095 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.617178 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:24.617633 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.618429 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.620245 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.620892 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.621021 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.621073 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.621402 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.621471 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260807 18:42:24.621594 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.616766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:35803 (local address 127.6.138.65:37525)
0807 18:42:24.616884 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.616890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.617025 (+ 135us) server_negotiation.cc:407] Connection header received
0807 18:42:24.617094 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.617097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.617146 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.617220 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:24.617633 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.618429 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.620245 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.620892 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.621025 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.621073 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.621402 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.621472 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:42:24.621630 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.616571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:45633 (local address 127.6.138.66:47839)
0807 18:42:24.616744 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:24.616757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:24.616863 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:24.617217 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:24.617221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:24.617237 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:24.617511 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.617519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.618547 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.618551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.620113 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.620126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.621023 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.621027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.621236 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.621294 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 18:42:24.621371 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 18:42:24.621467 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":323,"spinlock_wait_cycles":3200}
I20260807 18:42:24.621639 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.616635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37525 (local address 127.6.138.66:35803)
0807 18:42:24.616927 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:24.616942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:24.617029 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:24.617233 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:24.617236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:24.617247 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:24.617512 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.617519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.618547 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.618551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.620113 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:24.620126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.621023 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.621027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:24.621234 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.621287 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 18:42:24.621371 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 18:42:24.621467 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":276,"thread_start_us":112,"threads_started":1}
I20260807 18:42:24.622231 3021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309b300f0b264a8f99ce0655216fbd1a" candidate_uuid: "5dbafa8247224647bee74704bb35f5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" is_pre_election: true
I20260807 18:42:24.622287 3439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309b300f0b264a8f99ce0655216fbd1a" candidate_uuid: "5dbafa8247224647bee74704bb35f5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" is_pre_election: true
I20260807 18:42:24.622540 3021 raft_consensus.cc:2454] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dbafa8247224647bee74704bb35f5c8 in term 0.
I20260807 18:42:24.622552 3439 raft_consensus.cc:2454] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dbafa8247224647bee74704bb35f5c8 in term 0.
I20260807 18:42:24.623039 3127 leader_election.cc:302] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dbafa8247224647bee74704bb35f5c8, 9e4b76e1d15f4af9b0ffcb7ac4d413b6; no voters:
I20260807 18:42:24.623271 3531 raft_consensus.cc:2781] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:42:24.623368 3531 raft_consensus.cc:482] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:42:24.623413 3531 raft_consensus.cc:3037] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:24.624507 3531 raft_consensus.cc:504] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } }
I20260807 18:42:24.624840 3531 leader_election.cc:288] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [CANDIDATE]: Term 1 election: Requested vote from peers aba2e468ca5045ad8c5dfc2f4522665e (127.6.138.67:45633), 9e4b76e1d15f4af9b0ffcb7ac4d413b6 (127.6.138.65:37525)
I20260807 18:42:24.625198 3439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309b300f0b264a8f99ce0655216fbd1a" candidate_uuid: "5dbafa8247224647bee74704bb35f5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aba2e468ca5045ad8c5dfc2f4522665e"
I20260807 18:42:24.625277 3021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309b300f0b264a8f99ce0655216fbd1a" candidate_uuid: "5dbafa8247224647bee74704bb35f5c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6"
I20260807 18:42:24.625329 3439 raft_consensus.cc:3037] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:24.625427 3021 raft_consensus.cc:3037] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:24.626546 3439 raft_consensus.cc:2454] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dbafa8247224647bee74704bb35f5c8 in term 1.
I20260807 18:42:24.626631 3021 raft_consensus.cc:2454] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dbafa8247224647bee74704bb35f5c8 in term 1.
I20260807 18:42:24.626886 3130 leader_election.cc:302] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dbafa8247224647bee74704bb35f5c8, aba2e468ca5045ad8c5dfc2f4522665e; no voters:
I20260807 18:42:24.627064 3531 raft_consensus.cc:2781] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:24.627275 3531 raft_consensus.cc:686] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 1 LEADER]: Becoming Leader. State: Replica: 5dbafa8247224647bee74704bb35f5c8, State: Running, Role: LEADER
I20260807 18:42:24.627588 3531 consensus_queue.cc:237] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } }
I20260807 18:42:24.630856 2806 catalog_manager.cc:5495] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5dbafa8247224647bee74704bb35f5c8 (127.6.138.66). New cstate: current_term: 1 leader_uuid: "5dbafa8247224647bee74704bb35f5c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dbafa8247224647bee74704bb35f5c8" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 36647 } health_report { overall_health: HEALTHY } } }
I20260807 18:42:24.647580 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.643767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.6.138.66:36647)
0807 18:42:24.643919 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.643924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.643938 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:42:24.644112 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.644114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.644171 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.644306 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:42:24.644588 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.645107 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.645991 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.646217 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.646330 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.646665 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:42:24.647069 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:42:24.647231 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.647329 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.647368 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:42:24.654223 3021 raft_consensus.cc:1261] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6 [term 1 FOLLOWER]: Refusing update from remote peer 5dbafa8247224647bee74704bb35f5c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:24.654223 3439 raft_consensus.cc:1261] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [term 1 FOLLOWER]: Refusing update from remote peer 5dbafa8247224647bee74704bb35f5c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:24.655076 3534 consensus_queue.cc:1035] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aba2e468ca5045ad8c5dfc2f4522665e" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:24.655275 3531 consensus_queue.cc:1035] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4b76e1d15f4af9b0ffcb7ac4d413b6" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 37525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:24.662101 3537 mvcc.cc:205] Tried to move back new op lower bound from 7315980880494563328 to 7315980880397033472. Current Snapshot: MvccSnapshot[applied={T|T < 7315980880494563328}]
I20260807 18:42:24.664078 3538 mvcc.cc:205] Tried to move back new op lower bound from 7315980880494563328 to 7315980880397033472. Current Snapshot: MvccSnapshot[applied={T|T < 7315980880494563328}]
I20260807 18:42:24.666167 3540 mvcc.cc:205] Tried to move back new op lower bound from 7315980880494563328 to 7315980880397033472. Current Snapshot: MvccSnapshot[applied={T|T < 7315980880494563328}]
I20260807 18:42:24.672163 2806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:24.672417 2806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:24.675043 2806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6220dd087da64a57ae241511b2b563d9: Sending DeleteTablet for 3 replicas of tablet 309b300f0b264a8f99ce0655216fbd1a
I20260807 18:42:24.675902 3212 tablet_service.cc:1514] Processing DeleteTablet for tablet 309b300f0b264a8f99ce0655216fbd1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:24 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:36476
I20260807 18:42:24.675906 2987 tablet_service.cc:1514] Processing DeleteTablet for tablet 309b300f0b264a8f99ce0655216fbd1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:24 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:55518
I20260807 18:42:24.676486 3409 tablet_service.cc:1514] Processing DeleteTablet for tablet 309b300f0b264a8f99ce0655216fbd1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:24 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:57404
I20260807 18:42:24.676944 3550 tablet_replica.cc:330] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: stopping tablet replica
I20260807 18:42:24.677111 3554 tablet_replica.cc:330] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: stopping tablet replica
I20260807 18:42:24.677255 3552 tablet_replica.cc:330] T 309b300f0b264a8f99ce0655216fbd1a P 9e4b76e1d15f4af9b0ffcb7ac4d413b6: stopping tablet replica
I20260807 18:42:24.677510 3554 raft_consensus.cc:2229] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:42:24.677531 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 2903
I20260807 18:42:24.677855 3550 raft_consensus.cc:2229] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:42:24.678332 3550 raft_consensus.cc:2258] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:24.682157 3550 ts_tablet_manager.cc:1905] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:24.685102 3550 ts_tablet_manager.cc:1918] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:24.685271 3550 log.cc:1196] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/309b300f0b264a8f99ce0655216fbd1a
I20260807 18:42:24.685701 3550 ts_tablet_manager.cc:1939] T 309b300f0b264a8f99ce0655216fbd1a P 5dbafa8247224647bee74704bb35f5c8: Deleting consensus metadata
I20260807 18:42:24.687078 2789 catalog_manager.cc:4841] TS 5dbafa8247224647bee74704bb35f5c8 (127.6.138.66:36647): tablet 309b300f0b264a8f99ce0655216fbd1a (table test-table [id=78a316da22634bf9b85a98862d70c5c9]) successfully deleted
I20260807 18:42:24.687428 3554 raft_consensus.cc:2258] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:24.689248 3554 ts_tablet_manager.cc:1905] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:24.691946 3554 ts_tablet_manager.cc:1918] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:24.692063 3554 log.cc:1196] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/309b300f0b264a8f99ce0655216fbd1a
I20260807 18:42:24.692483 3554 ts_tablet_manager.cc:1939] T 309b300f0b264a8f99ce0655216fbd1a P aba2e468ca5045ad8c5dfc2f4522665e: Deleting consensus metadata
W20260807 18:42:24.692698 2788 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:37525 (error 108)
I20260807 18:42:24.693632 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 3102
I20260807 18:42:24.693840 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.693526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:37525 (local address 127.0.0.1:55530)
0807 18:42:24.693673 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:24.693753 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:37525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:42:24.694149 2788 catalog_manager.cc:4584] TS 9e4b76e1d15f4af9b0ffcb7ac4d413b6 (127.6.138.65:37525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 309b300f0b264a8f99ce0655216fbd1a: Network error: Client connection negotiation failed: client connection to 127.6.138.65:37525: connect: Connection refused (error 111)
I20260807 18:42:24.696014 2790 catalog_manager.cc:4841] TS aba2e468ca5045ad8c5dfc2f4522665e (127.6.138.67:45633): tablet 309b300f0b264a8f99ce0655216fbd1a (table test-table [id=78a316da22634bf9b85a98862d70c5c9]) successfully deleted
I20260807 18:42:24.705909 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 3321
I20260807 18:42:24.717386 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 2762
2026-08-07T18:42:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2663 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T18:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:24Z Disabled control of system clock
I20260807 18:42:24.755139 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:33139
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:44099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:33139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:42:24.868068 3559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:24.868350 3559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:24.868386 3559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:24.872439 3559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:24.872493 3559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:24.872514 3559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:24.872531 3559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:24.872546 3559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:24.877185 3559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:44099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:33139
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:33139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:24.878185 3559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:24.879527 3559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:24.888302 3564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:24.888298 3572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:24.888406 3568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:24.888679 3559 server_base.cc:1034] running on GCE node
I20260807 18:42:24.889468 3559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:24.890599 3559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:24.891794 3559 hybrid_clock.cc:648] HybridClock initialized: now 1786128144891778 us; error 33 us; skew 500 ppm
I20260807 18:42:24.893831 3559 webserver.cc:460] Webserver started at http://127.6.138.126:36953/ using document root <none> and password file <none>
I20260807 18:42:24.894425 3559 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:24.894529 3559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:24.894773 3559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:24.896899 3559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "2b159338172849568a9eac597c876cd9"
format_stamp: "Formatted at 2026-08-07 18:42:24 on dist-test-slave-2d19"
server_key: "9f1c7cde3aa5e022577625a056c31e21"
server_key_iv: "23f8b96e78e87efa8976a956a723122b"
server_key_version: "encryptionkey@0"
I20260807 18:42:24.897466 3559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "2b159338172849568a9eac597c876cd9"
format_stamp: "Formatted at 2026-08-07 18:42:24 on dist-test-slave-2d19"
server_key: "9f1c7cde3aa5e022577625a056c31e21"
server_key_iv: "23f8b96e78e87efa8976a956a723122b"
server_key_version: "encryptionkey@0"
I20260807 18:42:24.901170 3559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:42:24.903503 3578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:24.904731 3559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:24.904892 3559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "2b159338172849568a9eac597c876cd9"
format_stamp: "Formatted at 2026-08-07 18:42:24 on dist-test-slave-2d19"
server_key: "9f1c7cde3aa5e022577625a056c31e21"
server_key_iv: "23f8b96e78e87efa8976a956a723122b"
server_key_version: "encryptionkey@0"
I20260807 18:42:24.905004 3559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:24.923902 3559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:24.924655 3559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:24.924860 3559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:24.931885 3559 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:33139
I20260807 18:42:24.931890 3652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:33139 every 8 connection(s)
I20260807 18:42:24.932868 3559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:24.936036 3653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:24.941551 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 3559
I20260807 18:42:24.941602 3653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9: Bootstrap starting.
I20260807 18:42:24.941704 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:24.941992 6697 external_mini_cluster.cc:1428] Setting key b53656f4108fca087d5c0f8a7ce9340b
I20260807 18:42:24.944326 3653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:24.945086 3653 log.cc:824] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:24.946993 3653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9: No bootstrap required, opened a new log
I20260807 18:42:24.950543 3653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b159338172849568a9eac597c876cd9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33139 } }
I20260807 18:42:24.950832 3653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:24.950913 3653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b159338172849568a9eac597c876cd9, State: Initialized, Role: FOLLOWER
I20260807 18:42:24.951383 3653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b159338172849568a9eac597c876cd9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33139 } }
I20260807 18:42:24.951511 3653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:24.951606 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:24.943375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40022 (local address 127.6.138.126:33139)
0807 18:42:24.944087 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:24.944103 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:24.944139 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:42:24.944898 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:24.944918 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:24.945374 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:24.945722 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:24.946163 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.947176 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.948073 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:24.948623 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:24.948800 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:24.949700 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:24.949711 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:24.949725 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:24.950118 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:24.950179 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:24.950201 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:24.950313 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:24.950436 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 18:42:24.950711 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":150,"threads_started":1}
I20260807 18:42:24.951597 3653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:24.951799 3653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:24.952993 3653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b159338172849568a9eac597c876cd9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33139 } }
I20260807 18:42:24.953351 3653 leader_election.cc:302] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b159338172849568a9eac597c876cd9; no voters:
I20260807 18:42:24.953683 3653 leader_election.cc:288] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:24.953794 3661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:24.954070 3661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [term 1 LEADER]: Becoming Leader. State: Replica: 2b159338172849568a9eac597c876cd9, State: Running, Role: LEADER
I20260807 18:42:24.954378 3661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b159338172849568a9eac597c876cd9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33139 } }
I20260807 18:42:24.954902 3653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:24.956197 3662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b159338172849568a9eac597c876cd9. Latest consensus state: current_term: 1 leader_uuid: "2b159338172849568a9eac597c876cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b159338172849568a9eac597c876cd9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33139 } } }
I20260807 18:42:24.956197 3663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b159338172849568a9eac597c876cd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b159338172849568a9eac597c876cd9" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 33139 } } }
I20260807 18:42:24.956398 3663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:24.956411 3662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:24.956825 3676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:24.959631 3676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:24.965057 3676 catalog_manager.cc:1347] Generated new cluster ID: c74d9c41d6e546528728274452edaa28
I20260807 18:42:24.965137 3676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:24.981052 3676 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:24.981925 3676 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:24.987703 3676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9: Generated new TSK 0
I20260807 18:42:24.988466 3676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:24.998492 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33139
--builtin_ntp_servers=127.6.138.84:44099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:42:25.126998 3686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:25.127254 3686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:25.127291 3686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:25.131289 3686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:25.131350 3686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:25.131402 3686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:42:25.136032 3686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:44099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:25.137182 3686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:25.138572 3686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:25.148576 3691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:25.148595 3694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:25.148597 3692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:25.148825 3686 server_base.cc:1034] running on GCE node
I20260807 18:42:25.149245 3686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:25.149909 3686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:25.151081 3686 hybrid_clock.cc:648] HybridClock initialized: now 1786128145151065 us; error 28 us; skew 500 ppm
I20260807 18:42:25.153107 3686 webserver.cc:460] Webserver started at http://127.6.138.65:44697/ using document root <none> and password file <none>
I20260807 18:42:25.153715 3686 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:25.153780 3686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:25.154011 3686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:25.156231 3686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "df95ce1efc1b4b1f9f5080e6a696e69a"
format_stamp: "Formatted at 2026-08-07 18:42:25 on dist-test-slave-2d19"
server_key: "2ed2ade99853d577d632dc14e43013f2"
server_key_iv: "e35beceb7469b9b8a313a3fc3a389e24"
server_key_version: "encryptionkey@0"
I20260807 18:42:25.156870 3686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "df95ce1efc1b4b1f9f5080e6a696e69a"
format_stamp: "Formatted at 2026-08-07 18:42:25 on dist-test-slave-2d19"
server_key: "2ed2ade99853d577d632dc14e43013f2"
server_key_iv: "e35beceb7469b9b8a313a3fc3a389e24"
server_key_version: "encryptionkey@0"
I20260807 18:42:25.160662 3686 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:42:25.163170 3702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:25.164419 3686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:25.164565 3686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "df95ce1efc1b4b1f9f5080e6a696e69a"
format_stamp: "Formatted at 2026-08-07 18:42:25 on dist-test-slave-2d19"
server_key: "2ed2ade99853d577d632dc14e43013f2"
server_key_iv: "e35beceb7469b9b8a313a3fc3a389e24"
server_key_version: "encryptionkey@0"
I20260807 18:42:25.164685 3686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:25.199735 3686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:25.200557 3686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:25.200773 3686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:25.201416 3686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:25.202419 3686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:25.202502 3686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:25.202572 3686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:25.202625 3686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:25.211812 3686 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:45371
I20260807 18:42:25.211925 3869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:45371 every 8 connection(s)
I20260807 18:42:25.212857 3686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:42:25.217813 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 3686
I20260807 18:42:25.218055 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:42:25.218377 6697 external_mini_cluster.cc:1428] Setting key 04f887c3b279ff5dfc18f63ece1a39d8
I20260807 18:42:25.220475 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33139
--builtin_ntp_servers=127.6.138.84:44099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:42:25.223202 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.214814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:35691 (local address 127.6.138.126:33139)
0807 18:42:25.214955 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.214958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.216210 (+ 1252us) server_negotiation.cc:407] Connection header received
0807 18:42:25.217723 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.217726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.217786 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.217898 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:25.218997 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.219644 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.221082 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.221318 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.221438 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.222712 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:25.222715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:25.222717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:25.222730 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:25.222750 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:25.222758 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:25.222822 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.223036 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.223089 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:42:25.224395 3872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.215341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:33139 (local address 127.6.138.65:35691)
0807 18:42:25.216056 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:25.216098 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:25.217496 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:25.218092 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:25.218099 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:25.218223 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:25.218805 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.218824 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.219789 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.219792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.220898 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.220909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.221483 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.221487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.221666 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.222382 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:42:25.222386 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:25.222446 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:42:25.222537 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:25.222850 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:25.222863 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:25.222870 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:42:25.223011 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 18:42:25.223339 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":134,"threads_started":1}
I20260807 18:42:25.225625 3870 heartbeater.cc:347] Connected to a master server at 127.6.138.126:33139
I20260807 18:42:25.225885 3870 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:25.226423 3870 heartbeater.cc:511] Master 127.6.138.126:33139 requested a full tablet report, sending...
I20260807 18:42:25.227918 3603 ts_manager.cc:194] Registered new tserver with Master: df95ce1efc1b4b1f9f5080e6a696e69a (127.6.138.65:45371)
I20260807 18:42:25.229111 3603 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.65:35691
W20260807 18:42:25.341439 3874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:25.341708 3874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:25.341748 3874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:25.346107 3874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:25.346172 3874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:25.346268 3874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:42:25.351337 3874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:44099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:25.352617 3874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:25.354136 3874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:25.364604 3880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:25.364629 3883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:25.364614 3881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:25.364883 3874 server_base.cc:1034] running on GCE node
I20260807 18:42:25.365506 3874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:25.366261 3874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:25.367460 3874 hybrid_clock.cc:648] HybridClock initialized: now 1786128145367420 us; error 52 us; skew 500 ppm
I20260807 18:42:25.369716 3874 webserver.cc:460] Webserver started at http://127.6.138.66:38497/ using document root <none> and password file <none>
I20260807 18:42:25.370375 3874 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:25.370486 3874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:25.370795 3874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:25.373514 3874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "e86237a604884f2a8bd5beebbb80360d"
format_stamp: "Formatted at 2026-08-07 18:42:25 on dist-test-slave-2d19"
server_key: "d95cf6b800eb9c5770a50822b618d743"
server_key_iv: "f60a7a097ff6ebcab4f8c5fbe6e5b248"
server_key_version: "encryptionkey@0"
I20260807 18:42:25.374145 3874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "e86237a604884f2a8bd5beebbb80360d"
format_stamp: "Formatted at 2026-08-07 18:42:25 on dist-test-slave-2d19"
server_key: "d95cf6b800eb9c5770a50822b618d743"
server_key_iv: "f60a7a097ff6ebcab4f8c5fbe6e5b248"
server_key_version: "encryptionkey@0"
I20260807 18:42:25.378347 3874 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:42:25.381083 3891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:25.382336 3874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:25.382512 3874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "e86237a604884f2a8bd5beebbb80360d"
format_stamp: "Formatted at 2026-08-07 18:42:25 on dist-test-slave-2d19"
server_key: "d95cf6b800eb9c5770a50822b618d743"
server_key_iv: "f60a7a097ff6ebcab4f8c5fbe6e5b248"
server_key_version: "encryptionkey@0"
I20260807 18:42:25.382642 3874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:25.404706 3874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:25.405460 3874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:25.405678 3874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:25.406318 3874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:25.407310 3874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:25.407383 3874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:25.407485 3874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:25.407536 3874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:25.417457 3874 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:38243
I20260807 18:42:25.417485 4066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:38243 every 8 connection(s)
I20260807 18:42:25.418429 3874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:42:25.418998 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 3874
I20260807 18:42:25.419137 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:42:25.419461 6697 external_mini_cluster.cc:1428] Setting key f376dc922ac1b67d5a8f22089c32fd69
I20260807 18:42:25.421867 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33139
--builtin_ntp_servers=127.6.138.84:44099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:42:25.428936 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.420525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:38999 (local address 127.6.138.126:33139)
0807 18:42:25.420684 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.420689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.421887 (+ 1198us) server_negotiation.cc:407] Connection header received
0807 18:42:25.423396 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.423400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.423455 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.423536 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:25.424518 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.425172 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.426393 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.426788 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.426929 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.428241 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:25.428264 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:25.428270 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:25.428290 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:25.428319 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:25.428333 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:25.428419 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.428622 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.428700 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:42:25.429893 4070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.421018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:33139 (local address 127.6.138.66:38999)
0807 18:42:25.421738 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:25.421785 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:25.423177 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:25.423711 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:25.423718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:25.423804 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:25.424361 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.424378 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.425322 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.425325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.426243 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.426253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.426949 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.426952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.427109 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.427932 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:42:25.427934 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:25.427988 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:42:25.428057 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:25.428456 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:25.428466 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:25.428471 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:42:25.428584 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:42:25.428827 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":171,"threads_started":1}
I20260807 18:42:25.431185 4067 heartbeater.cc:347] Connected to a master server at 127.6.138.126:33139
I20260807 18:42:25.431473 4067 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:25.432219 4067 heartbeater.cc:511] Master 127.6.138.126:33139 requested a full tablet report, sending...
I20260807 18:42:25.433465 3603 ts_manager.cc:194] Registered new tserver with Master: e86237a604884f2a8bd5beebbb80360d (127.6.138.66:38243)
I20260807 18:42:25.434088 3603 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.66:38999
W20260807 18:42:25.541576 4072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:25.541893 4072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:25.541973 4072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:25.546186 4072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:25.546273 4072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:25.546387 4072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:42:25.551468 4072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:44099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:33139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:25.552759 4072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:25.554207 4072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:25.565714 4079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:25.565714 4082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:25.565735 4080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:25.566082 4072 server_base.cc:1034] running on GCE node
I20260807 18:42:25.566582 4072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:25.567286 4072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:25.568467 4072 hybrid_clock.cc:648] HybridClock initialized: now 1786128145568457 us; error 40 us; skew 500 ppm
I20260807 18:42:25.570657 4072 webserver.cc:460] Webserver started at http://127.6.138.67:45033/ using document root <none> and password file <none>
I20260807 18:42:25.571293 4072 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:25.571393 4072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:25.571661 4072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:25.573874 4072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "a2a05fff6dcf4da0b5ff714c3442b332"
format_stamp: "Formatted at 2026-08-07 18:42:25 on dist-test-slave-2d19"
server_key: "05a7e0014cd86e3335d302d26ddb0972"
server_key_iv: "1de0451e1e99853396cece39da3a621f"
server_key_version: "encryptionkey@0"
I20260807 18:42:25.574488 4072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "a2a05fff6dcf4da0b5ff714c3442b332"
format_stamp: "Formatted at 2026-08-07 18:42:25 on dist-test-slave-2d19"
server_key: "05a7e0014cd86e3335d302d26ddb0972"
server_key_iv: "1de0451e1e99853396cece39da3a621f"
server_key_version: "encryptionkey@0"
I20260807 18:42:25.578445 4072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 18:42:25.581086 4091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:25.582343 4072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:42:25.582487 4072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "a2a05fff6dcf4da0b5ff714c3442b332"
format_stamp: "Formatted at 2026-08-07 18:42:25 on dist-test-slave-2d19"
server_key: "05a7e0014cd86e3335d302d26ddb0972"
server_key_iv: "1de0451e1e99853396cece39da3a621f"
server_key_version: "encryptionkey@0"
I20260807 18:42:25.582612 4072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:25.593988 4072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:25.594707 4072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:25.594918 4072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:25.595525 4072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:25.596546 4072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:25.596616 4072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:25.596689 4072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:25.596743 4072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:25.605849 4072 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:36881
I20260807 18:42:25.605887 4248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:36881 every 8 connection(s)
I20260807 18:42:25.606789 4072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:42:25.610059 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 4072
I20260807 18:42:25.610170 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:42:25.610459 6697 external_mini_cluster.cc:1428] Setting key 2f8dca2b66f244191ff928f847f12358
I20260807 18:42:25.616837 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.608787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:50821 (local address 127.6.138.126:33139)
0807 18:42:25.609026 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.609032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.610322 (+ 1290us) server_negotiation.cc:407] Connection header received
0807 18:42:25.611958 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.611961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.612019 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.612097 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:25.613105 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.613640 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.614820 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.615050 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.615169 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.616309 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:25.616313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:25.616316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:25.616331 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:25.616355 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:25.616364 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:25.616444 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.616628 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.616691 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:42:25.617688 4252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.609291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:33139 (local address 127.6.138.67:50821)
0807 18:42:25.610102 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:25.610189 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:25.611709 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:25.612286 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:25.612294 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:25.612378 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:25.612956 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.612971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.613772 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.613777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.614688 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.614697 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.615216 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.615220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.615393 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.616026 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:42:25.616028 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:25.616068 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:42:25.616133 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:25.616463 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:25.616476 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:25.616486 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:42:25.616631 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 18:42:25.616929 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":128,"threads_started":1}
I20260807 18:42:25.618767 4249 heartbeater.cc:347] Connected to a master server at 127.6.138.126:33139
I20260807 18:42:25.618992 4249 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:25.619470 4249 heartbeater.cc:511] Master 127.6.138.126:33139 requested a full tablet report, sending...
I20260807 18:42:25.620488 3603 ts_manager.cc:194] Registered new tserver with Master: a2a05fff6dcf4da0b5ff714c3442b332 (127.6.138.67:36881)
I20260807 18:42:25.621058 3603 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.67:50821
I20260807 18:42:25.624744 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:42:25.630930 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.626812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40024 (local address 127.6.138.126:33139)
0807 18:42:25.626988 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.626993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.627161 (+ 168us) server_negotiation.cc:407] Connection header received
0807 18:42:25.627314 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.627319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.627363 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.627453 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:25.627764 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.628333 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.629187 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.629399 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.629504 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.630385 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:25.630388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:25.630390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:25.630402 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:25.630420 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:25.630428 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:25.630571 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.630736 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.630778 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:42:25.633932 3603 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:42:25.637019 3603 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:42:25.651736 4268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.645781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.6.138.67:36881)
0807 18:42:25.646353 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.646367 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.646398 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:42:25.646495 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.646502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.646730 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.646895 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:25.647463 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.648643 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.650369 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.651018 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.651148 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.651347 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.651450 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.651563 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":209,"threads_started":1}
I20260807 18:42:25.651728 4273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.646372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:45371 (local address 127.0.0.1:50332)
0807 18:42:25.647018 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:25.647033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:25.647128 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:25.647694 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:25.647697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:25.647714 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:25.647799 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.647805 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.648964 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.648968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.650233 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.650243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.651240 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.651244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.651335 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.651368 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:42:25.651423 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 18:42:25.651571 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":120,"thread_start_us":90,"threads_started":1}
I20260807 18:42:25.652061 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.646771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.6.138.65:45371)
0807 18:42:25.647237 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.647250 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.647273 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:42:25.647355 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.647362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.647562 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.647722 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:25.647899 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.648842 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.650356 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.651093 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.651565 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.651716 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.651765 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.651858 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":111,"threads_started":1}
I20260807 18:42:25.652089 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.645634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:36881 (local address 127.0.0.1:47516)
0807 18:42:25.646012 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:25.646074 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:25.646215 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:25.647144 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:25.647151 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:25.647179 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:25.647314 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.647323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.648778 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.648782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.650227 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.650239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.651150 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.651153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.651262 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.651298 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:42:25.651420 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 18:42:25.651941 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":76,"thread_start_us":80,"threads_started":1}
I20260807 18:42:25.652549 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.646721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:38243 (local address 127.0.0.1:43198)
0807 18:42:25.647310 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:25.647321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:25.647392 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:25.647931 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:25.647936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:25.647956 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:25.648029 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.648035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.649318 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.649321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.650506 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.650516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.651766 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.651770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.651862 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.651901 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:42:25.651988 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 18:42:25.652060 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":55,"thread_start_us":91,"threads_started":1}
I20260807 18:42:25.652781 4277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.646895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.6.138.66:38243)
0807 18:42:25.647500 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.647514 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.647538 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:42:25.647610 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.647615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.647788 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.647928 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:25.648122 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.649189 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.650636 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.651406 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.651512 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.651699 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.652166 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.652290 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":128,"threads_started":1}
I20260807 18:42:25.655320 3969 tablet_service.cc:1467] Processing CreateTablet for tablet 20494a5573d143e4bbc69de2257be709 (DEFAULT_TABLE table=test-table [id=0506fbae3a8c43cdae29195fe9cb420c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:25.655339 4159 tablet_service.cc:1467] Processing CreateTablet for tablet 20494a5573d143e4bbc69de2257be709 (DEFAULT_TABLE table=test-table [id=0506fbae3a8c43cdae29195fe9cb420c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:25.655341 3771 tablet_service.cc:1467] Processing CreateTablet for tablet 20494a5573d143e4bbc69de2257be709 (DEFAULT_TABLE table=test-table [id=0506fbae3a8c43cdae29195fe9cb420c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:25.656405 3771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20494a5573d143e4bbc69de2257be709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:25.656405 4159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20494a5573d143e4bbc69de2257be709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:25.656404 3969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20494a5573d143e4bbc69de2257be709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:25.662776 4278 tablet_bootstrap.cc:492] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: Bootstrap starting.
I20260807 18:42:25.664492 4281 tablet_bootstrap.cc:492] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a: Bootstrap starting.
I20260807 18:42:25.664570 4285 tablet_bootstrap.cc:492] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: Bootstrap starting.
I20260807 18:42:25.665628 4278 tablet_bootstrap.cc:654] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:25.666483 4285 tablet_bootstrap.cc:654] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:25.666560 4278 log.cc:824] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:25.666637 4281 tablet_bootstrap.cc:654] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:25.667315 4285 log.cc:824] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:25.667482 4281 log.cc:824] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:25.669461 4281 tablet_bootstrap.cc:492] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a: No bootstrap required, opened a new log
I20260807 18:42:25.669472 4285 tablet_bootstrap.cc:492] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: No bootstrap required, opened a new log
I20260807 18:42:25.669687 4281 ts_tablet_manager.cc:1397] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:42:25.669693 4285 ts_tablet_manager.cc:1397] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:42:25.670156 4278 tablet_bootstrap.cc:492] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: No bootstrap required, opened a new log
I20260807 18:42:25.670353 4278 ts_tablet_manager.cc:1397] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 18:42:25.673199 4281 raft_consensus.cc:348] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } }
I20260807 18:42:25.673451 4281 raft_consensus.cc:374] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:25.673516 4281 raft_consensus.cc:729] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df95ce1efc1b4b1f9f5080e6a696e69a, State: Initialized, Role: FOLLOWER
I20260807 18:42:25.673986 4281 consensus_queue.cc:260] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } }
I20260807 18:42:25.674198 4278 raft_consensus.cc:348] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } }
I20260807 18:42:25.674466 4278 raft_consensus.cc:374] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:25.674530 4278 raft_consensus.cc:729] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e86237a604884f2a8bd5beebbb80360d, State: Initialized, Role: FOLLOWER
I20260807 18:42:25.674387 4285 raft_consensus.cc:348] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } }
I20260807 18:42:25.674616 4285 raft_consensus.cc:374] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:25.674669 4285 raft_consensus.cc:729] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a05fff6dcf4da0b5ff714c3442b332, State: Initialized, Role: FOLLOWER
I20260807 18:42:25.675150 4281 ts_tablet_manager.cc:1428] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 18:42:25.675086 4285 consensus_queue.cc:260] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } }
I20260807 18:42:25.675345 4278 consensus_queue.cc:260] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } }
I20260807 18:42:25.675537 3870 heartbeater.cc:503] Master 127.6.138.126:33139 was elected leader, sending a full tablet report...
I20260807 18:42:25.676036 4067 heartbeater.cc:503] Master 127.6.138.126:33139 was elected leader, sending a full tablet report...
I20260807 18:42:25.676144 4285 ts_tablet_manager.cc:1428] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:42:25.676457 4249 heartbeater.cc:503] Master 127.6.138.126:33139 was elected leader, sending a full tablet report...
I20260807 18:42:25.678966 4278 ts_tablet_manager.cc:1428] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260807 18:42:25.714089 3871 tablet.cc:2369] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:42:25.735695 4292 raft_consensus.cc:482] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:25.735963 4292 raft_consensus.cc:504] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } }
I20260807 18:42:25.737175 4292 leader_election.cc:288] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2a05fff6dcf4da0b5ff714c3442b332 (127.6.138.67:36881), e86237a604884f2a8bd5beebbb80360d (127.6.138.66:38243)
I20260807 18:42:25.741647 4268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.737576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:60831 (local address 127.6.138.67:36881)
0807 18:42:25.737716 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.737720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.737879 (+ 159us) server_negotiation.cc:407] Connection header received
0807 18:42:25.738127 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.738131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.738195 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.738252 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:25.738617 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.739151 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.740373 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.740827 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.740980 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.741031 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.741312 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.741451 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:42:25.742094 4277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.738040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:47869 (local address 127.6.138.66:38243)
0807 18:42:25.738184 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.738190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.738414 (+ 224us) server_negotiation.cc:407] Connection header received
0807 18:42:25.738601 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.738604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.738659 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.738747 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:25.739037 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.739528 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.740872 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.741445 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.741561 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.741605 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.741864 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.741956 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:42:25.742162 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.737459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:36881 (local address 127.6.138.65:60831)
0807 18:42:25.737766 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:25.737784 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:25.737924 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:25.738382 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:25.738385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:25.738401 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:25.738501 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.738504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.739277 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.739281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.740177 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.740190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.740999 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.741003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.741101 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.741127 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:42:25.741241 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 18:42:25.741307 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":136,"threads_started":1}
I20260807 18:42:25.742506 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.737913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:38243 (local address 127.6.138.65:47869)
0807 18:42:25.738309 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:25.738320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:25.738460 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:25.738758 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:25.738760 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:25.738776 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:25.738913 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.738918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.739643 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.739648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.740711 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:25.740726 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.741592 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.741596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:25.741698 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.741725 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:42:25.742290 (+ 565us) client_negotiation.cc:240] Negotiation successful
0807 18:42:25.742370 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":127,"threads_started":1}
I20260807 18:42:25.743278 3993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20494a5573d143e4bbc69de2257be709" candidate_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86237a604884f2a8bd5beebbb80360d" is_pre_election: true
I20260807 18:42:25.743363 4185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20494a5573d143e4bbc69de2257be709" candidate_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" is_pre_election: true
I20260807 18:42:25.743642 3993 raft_consensus.cc:2454] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df95ce1efc1b4b1f9f5080e6a696e69a in term 0.
I20260807 18:42:25.743678 4185 raft_consensus.cc:2454] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df95ce1efc1b4b1f9f5080e6a696e69a in term 0.
I20260807 18:42:25.744140 3705 leader_election.cc:302] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: df95ce1efc1b4b1f9f5080e6a696e69a, e86237a604884f2a8bd5beebbb80360d; no voters:
I20260807 18:42:25.744464 4292 raft_consensus.cc:2781] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:42:25.744562 4292 raft_consensus.cc:482] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:42:25.744596 4292 raft_consensus.cc:3037] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:25.745715 4292 raft_consensus.cc:504] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } }
I20260807 18:42:25.746070 4292 leader_election.cc:288] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [CANDIDATE]: Term 1 election: Requested vote from peers a2a05fff6dcf4da0b5ff714c3442b332 (127.6.138.67:36881), e86237a604884f2a8bd5beebbb80360d (127.6.138.66:38243)
I20260807 18:42:25.746486 4185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20494a5573d143e4bbc69de2257be709" candidate_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2a05fff6dcf4da0b5ff714c3442b332"
I20260807 18:42:25.746536 3993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20494a5573d143e4bbc69de2257be709" candidate_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e86237a604884f2a8bd5beebbb80360d"
I20260807 18:42:25.746606 4185 raft_consensus.cc:3037] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:25.746727 3993 raft_consensus.cc:3037] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:25.747743 4185 raft_consensus.cc:2454] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df95ce1efc1b4b1f9f5080e6a696e69a in term 1.
I20260807 18:42:25.747923 3993 raft_consensus.cc:2454] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df95ce1efc1b4b1f9f5080e6a696e69a in term 1.
I20260807 18:42:25.748122 3705 leader_election.cc:302] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2a05fff6dcf4da0b5ff714c3442b332, df95ce1efc1b4b1f9f5080e6a696e69a; no voters:
I20260807 18:42:25.748354 4292 raft_consensus.cc:2781] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:25.748596 4292 raft_consensus.cc:686] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 1 LEADER]: Becoming Leader. State: Replica: df95ce1efc1b4b1f9f5080e6a696e69a, State: Running, Role: LEADER
I20260807 18:42:25.748941 4292 consensus_queue.cc:237] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } }
I20260807 18:42:25.752075 3602 catalog_manager.cc:5495] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a reported cstate change: term changed from 0 to 1, leader changed from <none> to df95ce1efc1b4b1f9f5080e6a696e69a (127.6.138.65). New cstate: current_term: 1 leader_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df95ce1efc1b4b1f9f5080e6a696e69a" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 45371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 } health_report { overall_health: UNKNOWN } } }
I20260807 18:42:25.776333 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.772907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.6.138.65:45371)
0807 18:42:25.773046 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:25.773049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:25.773062 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:42:25.773133 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:25.773135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:25.773192 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:25.773281 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:42:25.773515 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.774014 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.774860 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:25.775079 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:25.775160 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:25.775423 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:42:25.775837 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:42:25.775947 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:25.776099 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 18:42:25.776134 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:42:25.782900 3993 raft_consensus.cc:1261] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [term 1 FOLLOWER]: Refusing update from remote peer df95ce1efc1b4b1f9f5080e6a696e69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:25.782925 4185 raft_consensus.cc:1261] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [term 1 FOLLOWER]: Refusing update from remote peer df95ce1efc1b4b1f9f5080e6a696e69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:25.783671 4292 consensus_queue.cc:1035] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e86237a604884f2a8bd5beebbb80360d" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:25.783906 4300 consensus_queue.cc:1035] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2a05fff6dcf4da0b5ff714c3442b332" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 36881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:25.791344 4303 mvcc.cc:205] Tried to move back new op lower bound from 7315980885119016960 to 7315980884990291968. Current Snapshot: MvccSnapshot[applied={T|T < 7315980885119016960}]
I20260807 18:42:25.795138 4305 mvcc.cc:205] Tried to move back new op lower bound from 7315980885119016960 to 7315980884990291968. Current Snapshot: MvccSnapshot[applied={T|T < 7315980885119016960}]
I20260807 18:42:25.800567 3602 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:25.800742 3602 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:25.803081 3602 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b159338172849568a9eac597c876cd9: Sending DeleteTablet for 3 replicas of tablet 20494a5573d143e4bbc69de2257be709
I20260807 18:42:25.803864 4159 tablet_service.cc:1514] Processing DeleteTablet for tablet 20494a5573d143e4bbc69de2257be709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:25 UTC) from {username='slave'} at 127.0.0.1:47516
I20260807 18:42:25.804052 3771 tablet_service.cc:1514] Processing DeleteTablet for tablet 20494a5573d143e4bbc69de2257be709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:25 UTC) from {username='slave'} at 127.0.0.1:50332
I20260807 18:42:25.804184 3969 tablet_service.cc:1514] Processing DeleteTablet for tablet 20494a5573d143e4bbc69de2257be709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:25 UTC) from {username='slave'} at 127.0.0.1:43198
I20260807 18:42:25.804587 4316 tablet_replica.cc:330] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: stopping tablet replica
I20260807 18:42:25.804766 4319 tablet_replica.cc:330] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a: stopping tablet replica
I20260807 18:42:25.804913 4316 raft_consensus.cc:2229] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:42:25.805064 4319 raft_consensus.cc:2229] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:42:25.805145 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 3686
I20260807 18:42:25.805222 4316 raft_consensus.cc:2258] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:25.805521 4319 raft_consensus.cc:2258] T 20494a5573d143e4bbc69de2257be709 P df95ce1efc1b4b1f9f5080e6a696e69a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:25.810002 4320 tablet_replica.cc:330] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: stopping tablet replica
I20260807 18:42:25.810390 4320 raft_consensus.cc:2229] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:42:25.810722 4320 raft_consensus.cc:2258] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:25.811142 4316 ts_tablet_manager.cc:1905] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:25.812054 4320 ts_tablet_manager.cc:1905] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:25.814375 4320 ts_tablet_manager.cc:1918] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:25.814541 4320 log.cc:1196] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/20494a5573d143e4bbc69de2257be709
I20260807 18:42:25.814880 4320 ts_tablet_manager.cc:1939] T 20494a5573d143e4bbc69de2257be709 P e86237a604884f2a8bd5beebbb80360d: Deleting consensus metadata
I20260807 18:42:25.814873 4316 ts_tablet_manager.cc:1918] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:25.814973 4316 log.cc:1196] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/20494a5573d143e4bbc69de2257be709
I20260807 18:42:25.815300 4316 ts_tablet_manager.cc:1939] T 20494a5573d143e4bbc69de2257be709 P a2a05fff6dcf4da0b5ff714c3442b332: Deleting consensus metadata
I20260807 18:42:25.816200 3583 catalog_manager.cc:4841] TS e86237a604884f2a8bd5beebbb80360d (127.6.138.66:38243): tablet 20494a5573d143e4bbc69de2257be709 (table test-table [id=0506fbae3a8c43cdae29195fe9cb420c]) successfully deleted
I20260807 18:42:25.816489 3583 catalog_manager.cc:4841] TS a2a05fff6dcf4da0b5ff714c3442b332 (127.6.138.67:36881): tablet 20494a5573d143e4bbc69de2257be709 (table test-table [id=0506fbae3a8c43cdae29195fe9cb420c]) successfully deleted
W20260807 18:42:25.817287 3583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:45371 (error 108)
I20260807 18:42:25.818132 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 3874
I20260807 18:42:25.818408 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:25.818058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:45371 (local address 127.0.0.1:50342)
0807 18:42:25.818217 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:25.818306 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:45371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:42:25.820997 3583 catalog_manager.cc:4584] TS df95ce1efc1b4b1f9f5080e6a696e69a (127.6.138.65:45371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20494a5573d143e4bbc69de2257be709: Network error: Client connection negotiation failed: client connection to 127.6.138.65:45371: connect: Connection refused (error 111)
I20260807 18:42:25.843837 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 4072
I20260807 18:42:25.855077 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 3559
2026-08-07T18:42:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1141 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:25 dist-test-slave-2d19 krb5kdc[4325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:25 dist-test-slave-2d19 krb5kdc[4325](info): set up 2 sockets
Aug 07 18:42:25 dist-test-slave-2d19 krb5kdc[4325](info): commencing operation
krb5kdc: starting...
W20260807 18:42:27.132179 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:27 dist-test-slave-2d19 krb5kdc[4325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:42:27.267596 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:41229
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:40451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:41229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:27.383241 4345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:27.383543 4345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:27.383585 4345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:27.387717 4345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:27.387790 4345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:27.387810 4345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:27.387827 4345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:27.387842 4345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:27.393436 4345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:40451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:41229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:41229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:27.394635 4345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:27.395710 4345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:27.404625 4353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:27.404683 4356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:27.404625 4354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:27.404901 4345 server_base.cc:1034] running on GCE node
I20260807 18:42:27.405503 4345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:27.406635 4345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:27.407826 4345 hybrid_clock.cc:648] HybridClock initialized: now 1786128147407817 us; error 30 us; skew 500 ppm
Aug 07 18:42:27 dist-test-slave-2d19 krb5kdc[4325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128147, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:27.410615 4345 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:27.411444 4345 webserver.cc:460] Webserver started at http://127.6.138.126:34837/ using document root <none> and password file <none>
I20260807 18:42:27.411980 4345 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:27.412050 4345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:27.412214 4345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:27.414361 4345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "573a200c5076416d807f4d2ff89d3faa"
format_stamp: "Formatted at 2026-08-07 18:42:27 on dist-test-slave-2d19"
server_key: "c76a30d674f658d5b870e010750b7a3d"
server_key_iv: "220e5b728211390301912fbc4fe566c8"
server_key_version: "encryptionkey@0"
I20260807 18:42:27.414865 4345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "573a200c5076416d807f4d2ff89d3faa"
format_stamp: "Formatted at 2026-08-07 18:42:27 on dist-test-slave-2d19"
server_key: "c76a30d674f658d5b870e010750b7a3d"
server_key_iv: "220e5b728211390301912fbc4fe566c8"
server_key_version: "encryptionkey@0"
I20260807 18:42:27.418527 4345 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 18:42:27.421001 4365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:27.422260 4345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:27.422405 4345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "573a200c5076416d807f4d2ff89d3faa"
format_stamp: "Formatted at 2026-08-07 18:42:27 on dist-test-slave-2d19"
server_key: "c76a30d674f658d5b870e010750b7a3d"
server_key_iv: "220e5b728211390301912fbc4fe566c8"
server_key_version: "encryptionkey@0"
I20260807 18:42:27.422523 4345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:27.436317 4345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:27.439555 4345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:27.439790 4345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:27.447259 4345 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:41229
I20260807 18:42:27.447341 4443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:41229 every 8 connection(s)
I20260807 18:42:27.448342 4345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:27.451167 4444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:27.455056 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 4345
I20260807 18:42:27.455176 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:27.455477 6697 external_mini_cluster.cc:1428] Setting key ed401afc5edc72ff925aca3a5f215017
I20260807 18:42:27.457103 4444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa: Bootstrap starting.
I20260807 18:42:27.459630 4444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:27.460441 4444 log.cc:824] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:27.462623 4444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa: No bootstrap required, opened a new log
Aug 07 18:42:27 dist-test-slave-2d19 krb5kdc[4325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:27.466837 4444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573a200c5076416d807f4d2ff89d3faa" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41229 } }
I20260807 18:42:27.467058 4444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:27.467118 4444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573a200c5076416d807f4d2ff89d3faa, State: Initialized, Role: FOLLOWER
I20260807 18:42:27.467600 4444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573a200c5076416d807f4d2ff89d3faa" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41229 } }
I20260807 18:42:27.467761 4444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:27.467837 4444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:27.467967 4444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:27.468922 4444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573a200c5076416d807f4d2ff89d3faa" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41229 } }
I20260807 18:42:27.469271 4444 leader_election.cc:302] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 573a200c5076416d807f4d2ff89d3faa; no voters:
I20260807 18:42:27.469643 4444 leader_election.cc:288] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:27.470006 4452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:27.470201 4452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [term 1 LEADER]: Becoming Leader. State: Replica: 573a200c5076416d807f4d2ff89d3faa, State: Running, Role: LEADER
I20260807 18:42:27.470558 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:27.456885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.6.138.126:41229)
0807 18:42:27.457307 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:27.457324 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:27.457360 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:42:27.457988 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:27.458004 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:27.458330 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:27.458623 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:27.459581 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.460809 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.461781 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.462447 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.462614 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:27.465441 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:27.465446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:27.465459 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:27.465485 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:27.467338 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:27.467807 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:27.467813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:27.467819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:27.467983 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:27.468313 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:27.468317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:27.468319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:27.468714 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:27.468941 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:27.469229 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 18:42:27.469544 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":137,"threads_started":1}
I20260807 18:42:27.470516 4452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573a200c5076416d807f4d2ff89d3faa" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41229 } }
I20260807 18:42:27.470997 4444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:27.471738 4452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 573a200c5076416d807f4d2ff89d3faa. Latest consensus state: current_term: 1 leader_uuid: "573a200c5076416d807f4d2ff89d3faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573a200c5076416d807f4d2ff89d3faa" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41229 } } }
I20260807 18:42:27.471863 4452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:27.472643 4454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "573a200c5076416d807f4d2ff89d3faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573a200c5076416d807f4d2ff89d3faa" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41229 } } }
I20260807 18:42:27.472728 4454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:27.472990 4465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:27.476397 4465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:27.482018 4465 catalog_manager.cc:1347] Generated new cluster ID: 2ccd261e626c4f1e9d2e7828e891f11a
I20260807 18:42:27.482100 4465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:27.497308 4465 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:27.498253 4465 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:27.502203 4465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa: Generated new TSK 0
I20260807 18:42:27.502858 4465 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.138.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
Entry for principal kudu/127.6.138.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab.
I20260807 18:42:27.549465 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41229
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:40451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:27.685708 4481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:27.685987 4481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:27.686040 4481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:27.690030 4481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:27.690097 4481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:27.690157 4481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:42:27.695472 4481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:40451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.65.keytab
--principal=kudu/127.6.138.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:27.696553 4481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:27.697566 4481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:27.707592 4488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:27.707500 4487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:27.707495 4491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:27.707722 4481 server_base.cc:1034] running on GCE node
I20260807 18:42:27.708410 4481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:27.709064 4481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:27.710284 4481 hybrid_clock.cc:648] HybridClock initialized: now 1786128147710261 us; error 27 us; skew 500 ppm
Aug 07 18:42:27 dist-test-slave-2d19 krb5kdc[4325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128147, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:27.713384 4481 init.cc:377] Logged in from keytab as kudu/127.6.138.65@KRBTEST.COM (short username kudu)
I20260807 18:42:27.714319 4481 webserver.cc:460] Webserver started at http://127.6.138.65:33595/ using document root <none> and password file <none>
I20260807 18:42:27.714979 4481 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:27.715085 4481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:27.715337 4481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:27.717669 4481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "0cd853da70a94b33a19de36b0e294c1d"
format_stamp: "Formatted at 2026-08-07 18:42:27 on dist-test-slave-2d19"
server_key: "9a6accd41e71732ce155960acb652f3d"
server_key_iv: "3f9a2e543ab99fdfc696400ada7a11f7"
server_key_version: "encryptionkey@0"
I20260807 18:42:27.718282 4481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "0cd853da70a94b33a19de36b0e294c1d"
format_stamp: "Formatted at 2026-08-07 18:42:27 on dist-test-slave-2d19"
server_key: "9a6accd41e71732ce155960acb652f3d"
server_key_iv: "3f9a2e543ab99fdfc696400ada7a11f7"
server_key_version: "encryptionkey@0"
I20260807 18:42:27.722203 4481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:42:27.724774 4502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:27.726087 4481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:27.726217 4481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "0cd853da70a94b33a19de36b0e294c1d"
format_stamp: "Formatted at 2026-08-07 18:42:27 on dist-test-slave-2d19"
server_key: "9a6accd41e71732ce155960acb652f3d"
server_key_iv: "3f9a2e543ab99fdfc696400ada7a11f7"
server_key_version: "encryptionkey@0"
I20260807 18:42:27.726339 4481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:27.738866 4481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:27.741762 4481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:27.741971 4481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:27.742566 4481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:27.743389 4481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:27.743463 4481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:27.743542 4481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:27.743593 4481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:27.754010 4481 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:35565
I20260807 18:42:27.754030 4671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:35565 every 8 connection(s)
I20260807 18:42:27.754940 4481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:42:27.758111 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 4481
I20260807 18:42:27.758227 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:42:27.758539 6697 external_mini_cluster.cc:1428] Setting key b040e6fe345b5906cb7fbc20e14f0517
Aug 07 18:42:27 dist-test-slave-2d19 krb5kdc[4325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128147, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.65@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.66@KRBTEST.COM; defaulting to no policy
I20260807 18:42:27.769511 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:27.756824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:49907 (local address 127.6.138.126:41229)
0807 18:42:27.756964 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:27.756968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:27.757781 (+ 813us) server_negotiation.cc:407] Connection header received
0807 18:42:27.759314 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:27.759317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:27.759370 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:27.759464 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:27.760854 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.761393 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.762535 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.762775 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.762891 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:27.765670 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:27.765673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:27.765675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:27.765700 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:27.767311 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:27.767883 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:27.767886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:27.767888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:27.767976 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:27.768364 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:27.768368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:27.768371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:27.768555 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:27.768670 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:27.769193 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 18:42:27.769323 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:42:27.770411 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:27.757149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:41229 (local address 127.6.138.65:49907)
0807 18:42:27.757618 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:27.757662 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:27.759107 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:27.759637 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:27.759644 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:27.760125 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:27.760679 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:27.760695 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.761527 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.761531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:27.762397 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:27.762407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.762933 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.762937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:27.763107 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:27.763724 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:27.763750 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:27.765493 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:27.767479 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:27.767485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:27.767500 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:27.767766 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:27.768076 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:27.768079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:27.768081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:27.768223 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:27.768670 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:27.768678 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:27.768935 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 18:42:27.769132 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 18:42:27.769468 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
I20260807 18:42:27.771798 4672 heartbeater.cc:347] Connected to a master server at 127.6.138.126:41229
I20260807 18:42:27.772236 4672 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.66@KRBTEST.COM" created.
I20260807 18:42:27.772938 4672 heartbeater.cc:511] Master 127.6.138.126:41229 requested a full tablet report, sending...
I20260807 18:42:27.774521 4386 ts_manager.cc:194] Registered new tserver with Master: 0cd853da70a94b33a19de36b0e294c1d (127.6.138.65:35565)
I20260807 18:42:27.775738 4386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.65@KRBTEST.COM'} at 127.6.138.65:49907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
Entry for principal kudu/127.6.138.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab.
I20260807 18:42:27.791639 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41229
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:40451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:27.906237 4678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:27.906567 4678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:27.906630 4678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:27.912369 4678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:27.912447 4678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:27.912563 4678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:42:27.918452 4678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:40451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.66.keytab
--principal=kudu/127.6.138.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:27.919543 4678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:27.920650 4678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:27.930578 4690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:27.930621 4686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:27.930619 4684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:27.931114 4678 server_base.cc:1034] running on GCE node
I20260807 18:42:27.931520 4678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:27.932118 4678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:27.933306 4678 hybrid_clock.cc:648] HybridClock initialized: now 1786128147933271 us; error 45 us; skew 500 ppm
Aug 07 18:42:27 dist-test-slave-2d19 krb5kdc[4325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128147, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:27.936175 4678 init.cc:377] Logged in from keytab as kudu/127.6.138.66@KRBTEST.COM (short username kudu)
I20260807 18:42:27.937148 4678 webserver.cc:460] Webserver started at http://127.6.138.66:33371/ using document root <none> and password file <none>
I20260807 18:42:27.937737 4678 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:27.937834 4678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:27.938089 4678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:27.940181 4678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "30341186daba4293a6c474820352f56f"
format_stamp: "Formatted at 2026-08-07 18:42:27 on dist-test-slave-2d19"
server_key: "881f834c81849f6964807928cbd6a573"
server_key_iv: "ee865484d62fab6526026f90168cc0b4"
server_key_version: "encryptionkey@0"
I20260807 18:42:27.940814 4678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "30341186daba4293a6c474820352f56f"
format_stamp: "Formatted at 2026-08-07 18:42:27 on dist-test-slave-2d19"
server_key: "881f834c81849f6964807928cbd6a573"
server_key_iv: "ee865484d62fab6526026f90168cc0b4"
server_key_version: "encryptionkey@0"
I20260807 18:42:27.945330 4678 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:42:27.947845 4705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:27.949195 4678 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:27.949348 4678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "30341186daba4293a6c474820352f56f"
format_stamp: "Formatted at 2026-08-07 18:42:27 on dist-test-slave-2d19"
server_key: "881f834c81849f6964807928cbd6a573"
server_key_iv: "ee865484d62fab6526026f90168cc0b4"
server_key_version: "encryptionkey@0"
I20260807 18:42:27.949473 4678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:27.962836 4678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:27.966379 4678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:27.966594 4678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:27.967198 4678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:27.968058 4678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:27.968135 4678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:27.968209 4678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:27.968252 4678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:27.977982 4678 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:40713
I20260807 18:42:27.978021 4871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:40713 every 8 connection(s)
I20260807 18:42:27.978909 4678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:42:27.983669 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 4678
I20260807 18:42:27.983922 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:42:27.984230 6697 external_mini_cluster.cc:1428] Setting key a235a966abaeb5434eaa5302e1fc8f59
Aug 07 18:42:27 dist-test-slave-2d19 krb5kdc[4325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128147, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.66@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:27.994215 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:27.980769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:48797 (local address 127.6.138.126:41229)
0807 18:42:27.980935 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:27.980940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:27.981805 (+ 865us) server_negotiation.cc:407] Connection header received
0807 18:42:27.983125 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:27.983128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:27.983176 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:27.983257 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:27.984644 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.985286 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.986573 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.986808 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.986928 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:27.989861 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:27.989869 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:27.989875 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:27.989925 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:27.991744 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:27.992369 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:27.992373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:27.992375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:27.992480 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:27.992901 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:27.992907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:27.992910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:27.993198 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:27.993321 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:27.993858 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 18:42:27.994013 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":65}
I20260807 18:42:27.994990 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:27.981192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:41229 (local address 127.6.138.66:48797)
0807 18:42:27.981649 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:27.981693 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:27.982903 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:27.983430 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:27.983436 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:27.983786 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:27.984477 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:27.984494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.985423 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.985426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:27.986430 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:27.986441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:27.986971 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:27.986975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:27.987156 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:27.987833 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:27.987861 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:27.989668 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:27.991881 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:27.991886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:27.991899 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:27.992203 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:27.992623 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:27.992626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:27.992627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:27.992781 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:27.993321 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:27.993326 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:27.993584 (+ 258us) client_negotiation.cc:769] Sending connection context
0807 18:42:27.993746 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:42:27.994032 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.6.138.67@KRBTEST.COM; defaulting to no policy
I20260807 18:42:27.996412 4873 heartbeater.cc:347] Connected to a master server at 127.6.138.126:41229
I20260807 18:42:27.996690 4873 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:27.997260 4873 heartbeater.cc:511] Master 127.6.138.126:41229 requested a full tablet report, sending...
I20260807 18:42:27.998332 4386 ts_manager.cc:194] Registered new tserver with Master: 30341186daba4293a6c474820352f56f (127.6.138.66:40713)
I20260807 18:42:27.999094 4386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.66@KRBTEST.COM'} at 127.6.138.66:48797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
Entry for principal kudu/127.6.138.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab.
I20260807 18:42:28.015743 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41229
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:40451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:28.135488 4880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:28.135771 4880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:28.135851 4880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:28.140092 4880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:28.140169 4880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:28.140230 4880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:42:28.145880 4880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:40451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.67.keytab
--principal=kudu/127.6.138.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:28.147063 4880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:28.148145 4880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:28.159005 4891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:28.159005 4887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:28.159096 4880 server_base.cc:1034] running on GCE node
W20260807 18:42:28.159024 4888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:28.159646 4880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:28.160425 4880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:28.161609 4880 hybrid_clock.cc:648] HybridClock initialized: now 1786128148161574 us; error 49 us; skew 500 ppm
Aug 07 18:42:28 dist-test-slave-2d19 krb5kdc[4325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128148, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:28.164569 4880 init.cc:377] Logged in from keytab as kudu/127.6.138.67@KRBTEST.COM (short username kudu)
I20260807 18:42:28.165481 4880 webserver.cc:460] Webserver started at http://127.6.138.67:36255/ using document root <none> and password file <none>
I20260807 18:42:28.166092 4880 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:28.166193 4880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:28.166440 4880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:28.168630 4880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "3f6b26cbdf38425691d87bc6905b1074"
format_stamp: "Formatted at 2026-08-07 18:42:28 on dist-test-slave-2d19"
server_key: "2ecba733091271d97df68c6582f76f14"
server_key_iv: "d70965939d715771a6dbe013a9ab478b"
server_key_version: "encryptionkey@0"
I20260807 18:42:28.169224 4880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "3f6b26cbdf38425691d87bc6905b1074"
format_stamp: "Formatted at 2026-08-07 18:42:28 on dist-test-slave-2d19"
server_key: "2ecba733091271d97df68c6582f76f14"
server_key_iv: "d70965939d715771a6dbe013a9ab478b"
server_key_version: "encryptionkey@0"
I20260807 18:42:28.173458 4880 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:42:28.176227 4900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:28.177778 4880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:42:28.177937 4880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "3f6b26cbdf38425691d87bc6905b1074"
format_stamp: "Formatted at 2026-08-07 18:42:28 on dist-test-slave-2d19"
server_key: "2ecba733091271d97df68c6582f76f14"
server_key_iv: "d70965939d715771a6dbe013a9ab478b"
server_key_version: "encryptionkey@0"
I20260807 18:42:28.178081 4880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:28.194898 4880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:28.198031 4880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:28.198246 4880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:28.198984 4880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:28.199916 4880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:28.200011 4880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:28.200104 4880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:28.200160 4880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:28.209357 4880 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:38145
I20260807 18:42:28.209391 5070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:38145 every 8 connection(s)
I20260807 18:42:28.210270 4880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:42:28.213714 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 4880
I20260807 18:42:28.213852 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:42:28.214144 6697 external_mini_cluster.cc:1428] Setting key 04e18d1923385bf357dca64fa8dd453e
Aug 07 18:42:28 dist-test-slave-2d19 krb5kdc[4325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128148, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.67@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:28.224493 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.212216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:50153 (local address 127.6.138.126:41229)
0807 18:42:28.212409 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.212412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.213344 (+ 932us) server_negotiation.cc:407] Connection header received
0807 18:42:28.214898 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.214902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.214957 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.215073 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:28.216633 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.217156 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.218306 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.218525 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.218630 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.221065 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:28.221068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:28.221070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:28.221093 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:28.222453 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:28.222936 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:28.222939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:28.222940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:28.223013 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:28.223311 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:28.223313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:28.223314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:28.223480 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:28.223566 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.224155 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.224350 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 18:42:28.225381 5079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.212641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:41229 (local address 127.6.138.67:50153)
0807 18:42:28.213173 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:28.213217 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:28.214688 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:28.215309 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:28.215321 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:28.215837 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:28.216485 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.216516 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.217269 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.217272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.218173 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.218183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.218678 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.218681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.218818 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.219374 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:42:28.219391 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:28.220902 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:28.222572 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:28.222577 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:28.222588 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:28.222843 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:28.223093 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:42:28.223095 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:42:28.223096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:42:28.223228 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:42:28.223608 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:28.223613 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:28.223856 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 18:42:28.224021 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 18:42:28.224449 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":138,"threads_started":1}
I20260807 18:42:28.226698 5072 heartbeater.cc:347] Connected to a master server at 127.6.138.126:41229
I20260807 18:42:28.226980 5072 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:28.227629 5072 heartbeater.cc:511] Master 127.6.138.126:41229 requested a full tablet report, sending...
I20260807 18:42:28.228850 4386 ts_manager.cc:194] Registered new tserver with Master: 3f6b26cbdf38425691d87bc6905b1074 (127.6.138.67:38145)
I20260807 18:42:28.229411 4386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.138.67@KRBTEST.COM'} at 127.6.138.67:50153
I20260807 18:42:28.239971 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:42:28.249703 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.242106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.6.138.126:41229)
0807 18:42:28.242263 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.242268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.242455 (+ 187us) server_negotiation.cc:407] Connection header received
0807 18:42:28.242551 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.242553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.242594 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.242660 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:28.243345 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.243862 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.244706 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.244913 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.244990 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.246509 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:28.246511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:28.246513 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:28.246545 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:28.247828 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:28.248246 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:28.248300 (+ 54us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:28.248304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:28.248388 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:28.248661 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:28.248663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:28.248664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:28.248911 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:28.249018 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.249365 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.249539 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:42:28.252810 4386 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:42:28.256067 4386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:42:28.270572 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.264788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:40713 (local address 127.0.0.1:35470)
0807 18:42:28.265495 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:28.265573 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:28.265700 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:28.266469 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:28.266475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:28.266503 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:28.266656 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.266663 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.267865 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.267871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.269063 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.269072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.270041 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.270045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.270150 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.270193 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:42:28.270306 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:42:28.270418 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":76,"threads_started":1}
I20260807 18:42:28.270632 5091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.264911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35470 (local address 127.6.138.66:40713)
0807 18:42:28.265920 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.265932 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.265971 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:42:28.266061 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.266070 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.266293 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.266483 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:28.266794 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.267737 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.269183 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.269887 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.270032 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.270182 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.270354 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.270477 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":184,"threads_started":1}
I20260807 18:42:28.271169 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.264853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:38145 (local address 127.0.0.1:40468)
0807 18:42:28.265933 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:28.265949 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:28.266034 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:28.266664 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:28.266666 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:28.266681 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:28.266794 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.266800 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.268289 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.268293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.269445 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.269456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.270721 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.270726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.270840 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.270862 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:42:28.270975 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:42:28.271045 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"thread_start_us":121,"threads_started":1}
I20260807 18:42:28.271250 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.265263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.6.138.67:38145)
0807 18:42:28.265712 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.265725 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.266136 (+ 411us) server_negotiation.cc:407] Connection header received
0807 18:42:28.266222 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.266227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.266431 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.266602 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:28.266919 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.268093 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.269587 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.270407 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.270510 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.270646 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.271016 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.271107 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":129,"threads_started":1}
I20260807 18:42:28.272293 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.265983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42254 (local address 127.6.138.65:35565)
0807 18:42:28.266339 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.266356 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.266792 (+ 436us) server_negotiation.cc:407] Connection header received
0807 18:42:28.266885 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.266890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.267079 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.267234 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:28.267464 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.268760 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.269968 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.270961 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.271063 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.271198 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.271997 (+ 799us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.272099 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":83,"threads_started":1}
I20260807 18:42:28.272446 5093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.265560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:35565 (local address 127.0.0.1:42254)
0807 18:42:28.266336 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:28.266350 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:28.266456 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:28.267234 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:28.267237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:28.267255 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:28.267332 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.267338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.268912 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.268915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.269854 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.269861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.271529 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.271533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.271620 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.271642 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:42:28.271706 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 18:42:28.271770 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":86,"threads_started":1}
I20260807 18:42:28.274438 4975 tablet_service.cc:1467] Processing CreateTablet for tablet bf8ec125fdf0445f8c1dd99d0b687b29 (DEFAULT_TABLE table=test-table [id=28692534cfd443d291e330d7ff3c8cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:28.274466 4773 tablet_service.cc:1467] Processing CreateTablet for tablet bf8ec125fdf0445f8c1dd99d0b687b29 (DEFAULT_TABLE table=test-table [id=28692534cfd443d291e330d7ff3c8cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:28.275017 4573 tablet_service.cc:1467] Processing CreateTablet for tablet bf8ec125fdf0445f8c1dd99d0b687b29 (DEFAULT_TABLE table=test-table [id=28692534cfd443d291e330d7ff3c8cd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:28.275471 4975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf8ec125fdf0445f8c1dd99d0b687b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:28.275506 4773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf8ec125fdf0445f8c1dd99d0b687b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:28.275921 4573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf8ec125fdf0445f8c1dd99d0b687b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:28.281589 5095 tablet_bootstrap.cc:492] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: Bootstrap starting.
I20260807 18:42:28.283215 5098 tablet_bootstrap.cc:492] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d: Bootstrap starting.
I20260807 18:42:28.283239 5097 tablet_bootstrap.cc:492] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: Bootstrap starting.
I20260807 18:42:28.285221 5098 tablet_bootstrap.cc:654] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:28.285470 5095 tablet_bootstrap.cc:654] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:28.286041 5097 tablet_bootstrap.cc:654] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:28.286134 5098 log.cc:824] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:28.286840 5095 log.cc:824] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:28.286840 5097 log.cc:824] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:28.288344 5098 tablet_bootstrap.cc:492] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d: No bootstrap required, opened a new log
I20260807 18:42:28.288563 5098 ts_tablet_manager.cc:1397] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:42:28.288851 5097 tablet_bootstrap.cc:492] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: No bootstrap required, opened a new log
I20260807 18:42:28.288854 5095 tablet_bootstrap.cc:492] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: No bootstrap required, opened a new log
I20260807 18:42:28.289042 5097 ts_tablet_manager.cc:1397] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:42:28.289042 5095 ts_tablet_manager.cc:1397] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260807 18:42:28.292156 5098 raft_consensus.cc:348] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.292508 5098 raft_consensus.cc:374] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:28.292569 5098 raft_consensus.cc:729] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd853da70a94b33a19de36b0e294c1d, State: Initialized, Role: FOLLOWER
I20260807 18:42:28.293062 5098 consensus_queue.cc:260] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.292984 5097 raft_consensus.cc:348] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.292984 5095 raft_consensus.cc:348] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.293246 5097 raft_consensus.cc:374] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:28.293246 5095 raft_consensus.cc:374] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:28.293296 5097 raft_consensus.cc:729] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30341186daba4293a6c474820352f56f, State: Initialized, Role: FOLLOWER
I20260807 18:42:28.293296 5095 raft_consensus.cc:729] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f6b26cbdf38425691d87bc6905b1074, State: Initialized, Role: FOLLOWER
I20260807 18:42:28.293764 5095 consensus_queue.cc:260] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.293764 5097 consensus_queue.cc:260] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.294183 5098 ts_tablet_manager.cc:1428] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:42:28.294490 5072 heartbeater.cc:503] Master 127.6.138.126:41229 was elected leader, sending a full tablet report...
I20260807 18:42:28.294495 4672 heartbeater.cc:503] Master 127.6.138.126:41229 was elected leader, sending a full tablet report...
I20260807 18:42:28.294811 4873 heartbeater.cc:503] Master 127.6.138.126:41229 was elected leader, sending a full tablet report...
I20260807 18:42:28.295663 5097 ts_tablet_manager.cc:1428] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 18:42:28.296536 5095 ts_tablet_manager.cc:1428] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 18:42:28.428582 5106 raft_consensus.cc:482] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:28.428726 5108 raft_consensus.cc:482] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:28.428826 5106 raft_consensus.cc:504] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.428949 5108 raft_consensus.cc:504] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.430152 5106 leader_election.cc:288] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30341186daba4293a6c474820352f56f (127.6.138.66:40713), 3f6b26cbdf38425691d87bc6905b1074 (127.6.138.67:38145)
I20260807 18:42:28.430207 5108 leader_election.cc:288] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f6b26cbdf38425691d87bc6905b1074 (127.6.138.67:38145), 0cd853da70a94b33a19de36b0e294c1d (127.6.138.65:35565)
I20260807 18:42:28.435796 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.430425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:38145 (local address 127.6.138.66:56779)
0807 18:42:28.430561 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:28.430580 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:28.430681 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:28.431611 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:28.431615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:28.431632 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:28.431789 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.431796 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.433012 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.433015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.434457 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.434467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.435354 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.435357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.435465 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.435490 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:42:28.435555 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 18:42:28.435643 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":157,"spinlock_wait_cycles":1792}
I20260807 18:42:28.435909 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.430908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:54569 (local address 127.6.138.65:35565)
0807 18:42:28.431062 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.431067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.431249 (+ 182us) server_negotiation.cc:407] Connection header received
0807 18:42:28.431495 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.431499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.431560 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.431618 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:28.432155 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.432926 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.434623 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.435181 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.435262 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.435310 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.435708 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.435782 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:42:28.436013 5091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.430636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:35753 (local address 127.6.138.66:40713)
0807 18:42:28.430758 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.430762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.431437 (+ 675us) server_negotiation.cc:407] Connection header received
0807 18:42:28.431626 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.431629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.431690 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.431748 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:28.432150 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.432887 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.434594 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.435155 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.435255 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.435308 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.435695 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.435753 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":47,"spinlock_wait_cycles":2688}
I20260807 18:42:28.436380 5111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.430728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:35565 (local address 127.6.138.66:54569)
0807 18:42:28.431153 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:28.431170 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:28.431321 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:28.431822 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:28.431827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:28.431852 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:28.432009 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.432017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.433063 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.433066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.434460 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.434471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.435406 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.435409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.435527 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.435561 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:42:28.436101 (+ 540us) client_negotiation.cc:240] Negotiation successful
0807 18:42:28.436199 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":38,"spinlock_wait_cycles":768,"thread_start_us":119,"threads_started":1}
I20260807 18:42:28.436409 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.431106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:56779 (local address 127.6.138.67:38145)
0807 18:42:28.431227 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.431231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.431268 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:42:28.431329 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.431332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.431392 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.431476 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:28.431992 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.432864 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.434592 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.435166 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.435262 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.436138 (+ 876us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.436175 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.436242 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":842,"server-negotiator.queue_time_us":40}
I20260807 18:42:28.436950 5113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.431029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:38145 (local address 127.6.138.65:33907)
0807 18:42:28.431821 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:28.431834 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:28.431950 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:28.432481 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:28.432484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:28.432500 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:28.432609 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.432615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.433760 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.433764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.435044 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.435054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.436539 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.436543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.436647 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.436670 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:42:28.436721 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 18:42:28.436790 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":65,"spinlock_wait_cycles":3840,"thread_start_us":95,"threads_started":1}
I20260807 18:42:28.436956 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.430414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:40713 (local address 127.6.138.65:35753)
0807 18:42:28.431342 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:28.431356 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:28.431515 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:28.431844 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:28.431849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:28.431887 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:28.432024 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.432032 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.433027 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.433030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.434437 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:28.434450 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.435400 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.435404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:28.435538 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.435576 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:42:28.435816 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 18:42:28.435902 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"mutex_wait_us":34,"spinlock_wait_cycles":23296,"thread_start_us":183,"threads_started":1}
I20260807 18:42:28.437094 5004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ec125fdf0445f8c1dd99d0b687b29" candidate_uuid: "30341186daba4293a6c474820352f56f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f6b26cbdf38425691d87bc6905b1074" is_pre_election: true
I20260807 18:42:28.437269 4602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ec125fdf0445f8c1dd99d0b687b29" candidate_uuid: "30341186daba4293a6c474820352f56f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd853da70a94b33a19de36b0e294c1d" is_pre_election: true
I20260807 18:42:28.437458 5004 raft_consensus.cc:2454] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30341186daba4293a6c474820352f56f in term 0.
I20260807 18:42:28.437480 4602 raft_consensus.cc:2454] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30341186daba4293a6c474820352f56f in term 0.
I20260807 18:42:28.437525 5114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.431468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:33907 (local address 127.6.138.67:38145)
0807 18:42:28.432236 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.432245 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.432279 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:42:28.432320 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.432322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.432360 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.432705 (+ 345us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:28.432789 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.433630 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.435195 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.436386 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.437261 (+ 875us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.437303 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.437339 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.437398 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":692,"thread_start_us":103,"threads_started":1}
I20260807 18:42:28.437866 4802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ec125fdf0445f8c1dd99d0b687b29" candidate_uuid: "0cd853da70a94b33a19de36b0e294c1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30341186daba4293a6c474820352f56f" is_pre_election: true
I20260807 18:42:28.437927 5003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ec125fdf0445f8c1dd99d0b687b29" candidate_uuid: "0cd853da70a94b33a19de36b0e294c1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f6b26cbdf38425691d87bc6905b1074" is_pre_election: true
I20260807 18:42:28.437935 4706 leader_election.cc:302] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30341186daba4293a6c474820352f56f, 3f6b26cbdf38425691d87bc6905b1074; no voters:
I20260807 18:42:28.438027 5003 raft_consensus.cc:2454] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd853da70a94b33a19de36b0e294c1d in term 0.
I20260807 18:42:28.438073 4802 raft_consensus.cc:2454] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd853da70a94b33a19de36b0e294c1d in term 0.
I20260807 18:42:28.438185 5108 raft_consensus.cc:2781] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:42:28.438271 5108 raft_consensus.cc:482] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:42:28.438316 5108 raft_consensus.cc:3037] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:28.438395 4503 leader_election.cc:302] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd853da70a94b33a19de36b0e294c1d, 3f6b26cbdf38425691d87bc6905b1074; no voters:
I20260807 18:42:28.438836 5106 raft_consensus.cc:2781] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:42:28.438925 5106 raft_consensus.cc:482] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:42:28.438972 5106 raft_consensus.cc:3037] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:28.439754 5108 raft_consensus.cc:504] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.440203 5108 leader_election.cc:288] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [CANDIDATE]: Term 1 election: Requested vote from peers 3f6b26cbdf38425691d87bc6905b1074 (127.6.138.67:38145), 0cd853da70a94b33a19de36b0e294c1d (127.6.138.65:35565)
I20260807 18:42:28.440342 5106 raft_consensus.cc:504] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.440589 5003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ec125fdf0445f8c1dd99d0b687b29" candidate_uuid: "30341186daba4293a6c474820352f56f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f6b26cbdf38425691d87bc6905b1074"
I20260807 18:42:28.440701 5003 raft_consensus.cc:3037] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:28.440850 4602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ec125fdf0445f8c1dd99d0b687b29" candidate_uuid: "30341186daba4293a6c474820352f56f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd853da70a94b33a19de36b0e294c1d"
I20260807 18:42:28.441039 4602 raft_consensus.cc:2379] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 30341186daba4293a6c474820352f56f in current term 1: Already voted for candidate 0cd853da70a94b33a19de36b0e294c1d in this term.
I20260807 18:42:28.441082 5004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ec125fdf0445f8c1dd99d0b687b29" candidate_uuid: "0cd853da70a94b33a19de36b0e294c1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f6b26cbdf38425691d87bc6905b1074"
I20260807 18:42:28.441118 4802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ec125fdf0445f8c1dd99d0b687b29" candidate_uuid: "0cd853da70a94b33a19de36b0e294c1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30341186daba4293a6c474820352f56f"
I20260807 18:42:28.441303 4802 raft_consensus.cc:2379] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0cd853da70a94b33a19de36b0e294c1d in current term 1: Already voted for candidate 30341186daba4293a6c474820352f56f in this term.
I20260807 18:42:28.441665 5106 leader_election.cc:288] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [CANDIDATE]: Term 1 election: Requested vote from peers 30341186daba4293a6c474820352f56f (127.6.138.66:40713), 3f6b26cbdf38425691d87bc6905b1074 (127.6.138.67:38145)
I20260807 18:42:28.441838 4509 leader_election.cc:302] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cd853da70a94b33a19de36b0e294c1d; no voters: 30341186daba4293a6c474820352f56f, 3f6b26cbdf38425691d87bc6905b1074
I20260807 18:42:28.442037 5106 raft_consensus.cc:2726] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 18:42:28.442123 5003 raft_consensus.cc:2454] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30341186daba4293a6c474820352f56f in term 1.
I20260807 18:42:28.442415 4706 leader_election.cc:302] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30341186daba4293a6c474820352f56f, 3f6b26cbdf38425691d87bc6905b1074; no voters: 0cd853da70a94b33a19de36b0e294c1d
I20260807 18:42:28.442637 5108 raft_consensus.cc:2781] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:28.442898 5108 raft_consensus.cc:686] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 1 LEADER]: Becoming Leader. State: Replica: 30341186daba4293a6c474820352f56f, State: Running, Role: LEADER
I20260807 18:42:28.443262 5108 consensus_queue.cc:237] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } }
I20260807 18:42:28.446728 4386 catalog_manager.cc:5495] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f reported cstate change: term changed from 0 to 1, leader changed from <none> to 30341186daba4293a6c474820352f56f (127.6.138.66). New cstate: current_term: 1 leader_uuid: "30341186daba4293a6c474820352f56f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30341186daba4293a6c474820352f56f" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 40713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 } health_report { overall_health: UNKNOWN } } }
W20260807 18:42:28.461459 5075 tablet.cc:2369] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:42:28.467273 5091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.463836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35476 (local address 127.6.138.66:40713)
0807 18:42:28.463978 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.463982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.463996 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:42:28.464071 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.464073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.464124 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.464210 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:42:28.464545 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.465053 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.465923 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.466138 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.466215 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.466488 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:42:28.466879 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:42:28.466967 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.467122 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.467168 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:42:28.473567 4602 raft_consensus.cc:1261] T bf8ec125fdf0445f8c1dd99d0b687b29 P 0cd853da70a94b33a19de36b0e294c1d [term 1 FOLLOWER]: Refusing update from remote peer 30341186daba4293a6c474820352f56f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:28.473567 5003 raft_consensus.cc:1261] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 1 FOLLOWER]: Refusing update from remote peer 30341186daba4293a6c474820352f56f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:28.474263 5115 consensus_queue.cc:1035] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd853da70a94b33a19de36b0e294c1d" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:28.474479 5108 consensus_queue.cc:1035] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f6b26cbdf38425691d87bc6905b1074" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:42:28.480048 4874 tablet.cc:2369] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:42:28.483538 5121 mvcc.cc:205] Tried to move back new op lower bound from 7315980896139534336 to 7315980896026095616. Current Snapshot: MvccSnapshot[applied={T|T < 7315980896139534336}]
I20260807 18:42:28.492314 4386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:28.492517 4386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:28.494972 4386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 573a200c5076416d807f4d2ff89d3faa: Sending DeleteTablet for 3 replicas of tablet bf8ec125fdf0445f8c1dd99d0b687b29
I20260807 18:42:28.495784 4773 tablet_service.cc:1514] Processing DeleteTablet for tablet bf8ec125fdf0445f8c1dd99d0b687b29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:28 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:35470
I20260807 18:42:28.496069 4975 tablet_service.cc:1514] Processing DeleteTablet for tablet bf8ec125fdf0445f8c1dd99d0b687b29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:28 UTC) from {username='kudu', principal='kudu/127.6.138.126@KRBTEST.COM'} at 127.0.0.1:40468
I20260807 18:42:28.496372 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 4481
I20260807 18:42:28.508625 5137 tablet_replica.cc:330] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: stopping tablet replica
I20260807 18:42:28.509028 5137 raft_consensus.cc:2229] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:42:28.509395 5137 raft_consensus.cc:2258] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:28.512306 5137 ts_tablet_manager.cc:1905] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:28.514513 5133 tablet_replica.cc:330] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: stopping tablet replica
I20260807 18:42:28.514930 5133 raft_consensus.cc:2229] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:42:28.515398 5137 ts_tablet_manager.cc:1918] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:28.515515 5137 log.cc:1196] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/bf8ec125fdf0445f8c1dd99d0b687b29
I20260807 18:42:28.515679 5133 raft_consensus.cc:2258] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:28.515889 5137 ts_tablet_manager.cc:1939] T bf8ec125fdf0445f8c1dd99d0b687b29 P 3f6b26cbdf38425691d87bc6905b1074: Deleting consensus metadata
I20260807 18:42:28.517249 4366 catalog_manager.cc:4841] TS 3f6b26cbdf38425691d87bc6905b1074 (127.6.138.67:38145): tablet bf8ec125fdf0445f8c1dd99d0b687b29 (table test-table [id=28692534cfd443d291e330d7ff3c8cd4]) successfully deleted
I20260807 18:42:28.519121 5133 ts_tablet_manager.cc:1905] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:28.522989 5133 ts_tablet_manager.cc:1918] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:28.523118 5133 log.cc:1196] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/bf8ec125fdf0445f8c1dd99d0b687b29
I20260807 18:42:28.523516 5133 ts_tablet_manager.cc:1939] T bf8ec125fdf0445f8c1dd99d0b687b29 P 30341186daba4293a6c474820352f56f: Deleting consensus metadata
I20260807 18:42:28.524945 4368 catalog_manager.cc:4841] TS 30341186daba4293a6c474820352f56f (127.6.138.66:40713): tablet bf8ec125fdf0445f8c1dd99d0b687b29 (table test-table [id=28692534cfd443d291e330d7ff3c8cd4]) successfully deleted
W20260807 18:42:28.527981 4366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:35565 (error 108)
I20260807 18:42:28.528242 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 4678
I20260807 18:42:28.533051 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.532707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:35565 (local address 127.0.0.1:42270)
0807 18:42:28.532849 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:28.532939 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:35565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:42:28.533493 4366 catalog_manager.cc:4584] TS 0cd853da70a94b33a19de36b0e294c1d (127.6.138.65:35565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf8ec125fdf0445f8c1dd99d0b687b29: Network error: Client connection negotiation failed: client connection to 127.6.138.65:35565: connect: Connection refused (error 111)
I20260807 18:42:28.539196 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 4880
I20260807 18:42:28.554592 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 4345
2026-08-07T18:42:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2699 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T18:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:28Z Disabled control of system clock
I20260807 18:42:28.596974 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:35645
--webserver_interface=127.6.138.126
--webserver_port=0
--builtin_ntp_servers=127.6.138.84:46117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:35645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:42:28.711092 5145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:28.711371 5145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:28.711407 5145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:28.715565 5145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:28.715629 5145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:28.715646 5145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:28.715705 5145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:28.715734 5145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:28.720510 5145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:35645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:35645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:28.721629 5145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:28.723121 5145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:28.731539 5155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:28.731534 5156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:28.731534 5158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:28.731949 5145 server_base.cc:1034] running on GCE node
I20260807 18:42:28.732573 5145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:28.733726 5145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:28.734890 5145 hybrid_clock.cc:648] HybridClock initialized: now 1786128148734874 us; error 33 us; skew 500 ppm
I20260807 18:42:28.740860 5145 webserver.cc:460] Webserver started at http://127.6.138.126:44837/ using document root <none> and password file <none>
I20260807 18:42:28.741496 5145 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:28.741564 5145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:28.741808 5145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:28.743909 5145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c"
format_stamp: "Formatted at 2026-08-07 18:42:28 on dist-test-slave-2d19"
server_key: "df7633095503aea879af56865396e9dc"
server_key_iv: "eb3435b9426979613c4de0e4b732f7a4"
server_key_version: "encryptionkey@0"
I20260807 18:42:28.744536 5145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c"
format_stamp: "Formatted at 2026-08-07 18:42:28 on dist-test-slave-2d19"
server_key: "df7633095503aea879af56865396e9dc"
server_key_iv: "eb3435b9426979613c4de0e4b732f7a4"
server_key_version: "encryptionkey@0"
I20260807 18:42:28.748231 5145 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:42:28.750852 5166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:28.752506 5145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:42:28.752635 5145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c"
format_stamp: "Formatted at 2026-08-07 18:42:28 on dist-test-slave-2d19"
server_key: "df7633095503aea879af56865396e9dc"
server_key_iv: "eb3435b9426979613c4de0e4b732f7a4"
server_key_version: "encryptionkey@0"
I20260807 18:42:28.752772 5145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:28.810324 5145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:28.811105 5145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:28.811272 5145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:28.818558 5145 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:35645
I20260807 18:42:28.818564 5245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:35645 every 8 connection(s)
I20260807 18:42:28.819563 5145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:28.822815 5248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:28.824357 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 5145
I20260807 18:42:28.824483 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:28.824831 6697 external_mini_cluster.cc:1428] Setting key f55c19237f29848253857cac79bcc3f6
I20260807 18:42:28.829385 5248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c: Bootstrap starting.
I20260807 18:42:28.831631 5248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:28.832564 5248 log.cc:824] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:28.834568 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:28.826348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.6.138.126:35645)
0807 18:42:28.827019 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:28.827061 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:28.827098 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:42:28.827764 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:28.827782 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:28.828193 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:28.828593 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:28.829056 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.830102 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.830993 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:28.831466 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:28.831653 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:28.832590 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:28.832595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:28.832610 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:28.833123 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:28.833198 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:28.833228 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:28.833339 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:28.833530 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 18:42:28.833824 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":140,"threads_started":1}
I20260807 18:42:28.834729 5248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c: No bootstrap required, opened a new log
I20260807 18:42:28.838557 5248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35645 } }
I20260807 18:42:28.838801 5248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:28.838882 5248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d60e20fe7cd4b22b5d6ff3da326ed9c, State: Initialized, Role: FOLLOWER
I20260807 18:42:28.839300 5248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35645 } }
I20260807 18:42:28.839453 5248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:28.839545 5248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:28.839668 5248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:28.840759 5248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35645 } }
I20260807 18:42:28.841114 5248 leader_election.cc:302] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d60e20fe7cd4b22b5d6ff3da326ed9c; no voters:
I20260807 18:42:28.841454 5248 leader_election.cc:288] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:28.841585 5257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:28.841820 5257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [term 1 LEADER]: Becoming Leader. State: Replica: 6d60e20fe7cd4b22b5d6ff3da326ed9c, State: Running, Role: LEADER
I20260807 18:42:28.842160 5257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35645 } }
I20260807 18:42:28.842778 5248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:28.843941 5261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d60e20fe7cd4b22b5d6ff3da326ed9c. Latest consensus state: current_term: 1 leader_uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35645 } } }
I20260807 18:42:28.843947 5258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d60e20fe7cd4b22b5d6ff3da326ed9c" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 35645 } } }
I20260807 18:42:28.844213 5258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:28.844244 5261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:28.844666 5273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:28.847506 5273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:28.853466 5273 catalog_manager.cc:1347] Generated new cluster ID: cc998e8f7876494fb1122ca52d59e98a
I20260807 18:42:28.853547 5273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:28.860301 5273 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:28.861500 5273 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:28.880296 5273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c: Generated new TSK 0
I20260807 18:42:28.881215 5273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:28.896060 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.65:0
--local_ip_for_outbound_sockets=127.6.138.65
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35645
--builtin_ntp_servers=127.6.138.84:46117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:42:29.022068 5287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:29.022351 5287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:29.022433 5287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:29.026961 5287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:29.027026 5287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:29.027127 5287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.65
I20260807 18:42:29.032516 5287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.138.65
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.65
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:29.033911 5287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:29.035441 5287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:29.045897 5296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:29.045943 5295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:29.045897 5298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:29.046274 5287 server_base.cc:1034] running on GCE node
I20260807 18:42:29.046770 5287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:29.047503 5287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:29.048692 5287 hybrid_clock.cc:648] HybridClock initialized: now 1786128149048667 us; error 36 us; skew 500 ppm
I20260807 18:42:29.050598 5287 webserver.cc:460] Webserver started at http://127.6.138.65:41125/ using document root <none> and password file <none>
I20260807 18:42:29.051250 5287 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:29.051350 5287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:29.051611 5287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:29.053887 5287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/data/instance:
uuid: "3afb198b323a4ee6b6c15ab7226f4610"
format_stamp: "Formatted at 2026-08-07 18:42:29 on dist-test-slave-2d19"
server_key: "5648bca41dd213243c0265f78b101910"
server_key_iv: "af312ac0bf366775ff8b91fc914d4a06"
server_key_version: "encryptionkey@0"
I20260807 18:42:29.054492 5287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance:
uuid: "3afb198b323a4ee6b6c15ab7226f4610"
format_stamp: "Formatted at 2026-08-07 18:42:29 on dist-test-slave-2d19"
server_key: "5648bca41dd213243c0265f78b101910"
server_key_iv: "af312ac0bf366775ff8b91fc914d4a06"
server_key_version: "encryptionkey@0"
I20260807 18:42:29.058413 5287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:42:29.061280 5310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:29.062512 5287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:29.062659 5287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/wal
uuid: "3afb198b323a4ee6b6c15ab7226f4610"
format_stamp: "Formatted at 2026-08-07 18:42:29 on dist-test-slave-2d19"
server_key: "5648bca41dd213243c0265f78b101910"
server_key_iv: "af312ac0bf366775ff8b91fc914d4a06"
server_key_version: "encryptionkey@0"
I20260807 18:42:29.062785 5287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:29.093178 5287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:29.093988 5287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:29.094210 5287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:29.094852 5287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:29.095849 5287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:29.095927 5287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:29.095995 5287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:29.096040 5287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:29.105489 5287 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.65:34761
I20260807 18:42:29.105522 5490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.65:34761 every 8 connection(s)
I20260807 18:42:29.106416 5287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/data/info.pb
I20260807 18:42:29.114462 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 5287
I20260807 18:42:29.114631 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-0/wal/instance
I20260807 18:42:29.115020 6697 external_mini_cluster.cc:1428] Setting key 7c62968e37f8390e16284fdda13a333a
I20260807 18:42:29.116107 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.108340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.65:42199 (local address 127.6.138.126:35645)
0807 18:42:29.108480 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.108483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.109571 (+ 1088us) server_negotiation.cc:407] Connection header received
0807 18:42:29.111032 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.111036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.111101 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.111213 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:29.112131 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.112699 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.113876 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.114115 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.114229 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.115536 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:29.115540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:29.115543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:29.115555 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:29.115573 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:29.115581 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:29.115660 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.115877 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.115966 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:42:29.117043 5496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.108763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:35645 (local address 127.6.138.65:42199)
0807 18:42:29.109417 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:29.109463 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:29.110807 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:29.111400 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:29.111407 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:29.111490 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:29.111987 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.112000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.112849 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.112852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.113743 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.113753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.114286 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.114290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.114458 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.115240 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:42:29.115242 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:29.115293 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:42:29.115357 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:29.115664 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:29.115677 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:29.115683 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:42:29.115839 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 18:42:29.116094 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":107,"threads_started":1}
I20260807 18:42:29.117383 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.66:0
--local_ip_for_outbound_sockets=127.6.138.66
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35645
--builtin_ntp_servers=127.6.138.84:46117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:42:29.118278 5492 heartbeater.cc:347] Connected to a master server at 127.6.138.126:35645
I20260807 18:42:29.118535 5492 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:29.119038 5492 heartbeater.cc:511] Master 127.6.138.126:35645 requested a full tablet report, sending...
I20260807 18:42:29.120568 5187 ts_manager.cc:194] Registered new tserver with Master: 3afb198b323a4ee6b6c15ab7226f4610 (127.6.138.65:34761)
I20260807 18:42:29.121676 5187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.65:42199
W20260807 18:42:29.233748 5497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:29.234150 5497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:29.234216 5497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:29.240360 5497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:29.240448 5497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:29.240547 5497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.66
I20260807 18:42:29.248746 5497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.138.66
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.66
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:29.250854 5497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:29.252597 5497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:29.264973 5514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:29.266634 5510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:29.266733 5506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:29.267141 5497 server_base.cc:1034] running on GCE node
I20260807 18:42:29.267692 5497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:29.268551 5497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:29.269760 5497 hybrid_clock.cc:648] HybridClock initialized: now 1786128149269704 us; error 78 us; skew 500 ppm
I20260807 18:42:29.271639 5497 webserver.cc:460] Webserver started at http://127.6.138.66:35455/ using document root <none> and password file <none>
I20260807 18:42:29.272336 5497 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:29.272418 5497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:29.272672 5497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:29.274832 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/data/instance:
uuid: "abeb497278164496b5c7f813d5030d48"
format_stamp: "Formatted at 2026-08-07 18:42:29 on dist-test-slave-2d19"
server_key: "b780746cc444ef15a335b881989e7d4a"
server_key_iv: "4a4632ecd2e8cdb005d4033284625abe"
server_key_version: "encryptionkey@0"
I20260807 18:42:29.275418 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance:
uuid: "abeb497278164496b5c7f813d5030d48"
format_stamp: "Formatted at 2026-08-07 18:42:29 on dist-test-slave-2d19"
server_key: "b780746cc444ef15a335b881989e7d4a"
server_key_iv: "4a4632ecd2e8cdb005d4033284625abe"
server_key_version: "encryptionkey@0"
I20260807 18:42:29.279289 5497 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:42:29.281816 5525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:29.283079 5497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 18:42:29.283213 5497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/wal
uuid: "abeb497278164496b5c7f813d5030d48"
format_stamp: "Formatted at 2026-08-07 18:42:29 on dist-test-slave-2d19"
server_key: "b780746cc444ef15a335b881989e7d4a"
server_key_iv: "4a4632ecd2e8cdb005d4033284625abe"
server_key_version: "encryptionkey@0"
I20260807 18:42:29.283324 5497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:29.294780 5497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:29.295472 5497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:29.295679 5497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:29.296391 5497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:29.297333 5497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:29.297381 5497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:29.297451 5497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:29.297492 5497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:29.306766 5497 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.66:38403
I20260807 18:42:29.306792 5706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.66:38403 every 8 connection(s)
I20260807 18:42:29.307675 5497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/data/info.pb
I20260807 18:42:29.317270 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 5497
I20260807 18:42:29.317443 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/wal/instance
I20260807 18:42:29.317610 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.309599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.66:52223 (local address 127.6.138.126:35645)
0807 18:42:29.309775 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.309780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.310878 (+ 1098us) server_negotiation.cc:407] Connection header received
0807 18:42:29.312609 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.312611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.312659 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.312739 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:29.313771 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.314295 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.315501 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.315705 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.315783 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.317040 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:29.317044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:29.317047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:29.317062 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:29.317088 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:29.317101 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:29.317195 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.317337 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.317412 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:42:29.317819 6697 external_mini_cluster.cc:1428] Setting key 9daa5e46ee6ec53f891f92abb2b45760
I20260807 18:42:29.318631 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.310007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:35645 (local address 127.6.138.66:52223)
0807 18:42:29.310713 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:29.310758 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:29.312390 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:29.312958 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:29.312967 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:29.313061 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:29.313627 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.313642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.314410 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.314412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.315381 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.315391 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.315857 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.315861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.316034 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.316758 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:42:29.316759 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:29.316800 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:42:29.316862 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:29.317194 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:29.317201 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:29.317206 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:42:29.317302 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:42:29.317602 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":110,"threads_started":1}
I20260807 18:42:29.319702 5707 heartbeater.cc:347] Connected to a master server at 127.6.138.126:35645
I20260807 18:42:29.319955 5707 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:29.319978 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.138.67:0
--local_ip_for_outbound_sockets=127.6.138.67
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35645
--builtin_ntp_servers=127.6.138.84:46117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:42:29.320573 5707 heartbeater.cc:511] Master 127.6.138.126:35645 requested a full tablet report, sending...
I20260807 18:42:29.321563 5187 ts_manager.cc:194] Registered new tserver with Master: abeb497278164496b5c7f813d5030d48 (127.6.138.66:38403)
I20260807 18:42:29.322108 5187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.66:52223
W20260807 18:42:29.438431 5712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:29.438750 5712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:29.438822 5712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:29.442974 5712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:29.443063 5712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:29.443181 5712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.138.67
I20260807 18:42:29.448118 5712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:46117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.138.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.138.67
--webserver_port=0
--tserver_master_addrs=127.6.138.126:35645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.138.67
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:29.449316 5712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:29.450807 5712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:29.461103 5730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:29.461045 5728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:29.461045 5722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:29.461438 5712 server_base.cc:1034] running on GCE node
I20260807 18:42:29.461807 5712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:29.462482 5712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:29.463673 5712 hybrid_clock.cc:648] HybridClock initialized: now 1786128149463639 us; error 43 us; skew 500 ppm
I20260807 18:42:29.466041 5712 webserver.cc:460] Webserver started at http://127.6.138.67:42771/ using document root <none> and password file <none>
I20260807 18:42:29.466605 5712 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:29.466666 5712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:29.466892 5712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:29.469133 5712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/data/instance:
uuid: "9c37551485d1447a8851d6d73a0012f7"
format_stamp: "Formatted at 2026-08-07 18:42:29 on dist-test-slave-2d19"
server_key: "18c0531af13e6930798089765663169a"
server_key_iv: "4795304d81e47e40a69ac95ad0990734"
server_key_version: "encryptionkey@0"
I20260807 18:42:29.469704 5712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance:
uuid: "9c37551485d1447a8851d6d73a0012f7"
format_stamp: "Formatted at 2026-08-07 18:42:29 on dist-test-slave-2d19"
server_key: "18c0531af13e6930798089765663169a"
server_key_iv: "4795304d81e47e40a69ac95ad0990734"
server_key_version: "encryptionkey@0"
I20260807 18:42:29.473685 5712 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:42:29.476233 5744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:29.477612 5712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:29.477777 5712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/wal
uuid: "9c37551485d1447a8851d6d73a0012f7"
format_stamp: "Formatted at 2026-08-07 18:42:29 on dist-test-slave-2d19"
server_key: "18c0531af13e6930798089765663169a"
server_key_iv: "4795304d81e47e40a69ac95ad0990734"
server_key_version: "encryptionkey@0"
I20260807 18:42:29.477903 5712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:29.493094 5712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:29.493845 5712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:29.494066 5712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:29.494702 5712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:42:29.495741 5712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:42:29.495819 5712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:29.495898 5712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:42:29.495942 5712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:29.504796 5712 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.67:45945
I20260807 18:42:29.504832 5934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.67:45945 every 8 connection(s)
I20260807 18:42:29.505723 5712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/data/info.pb
I20260807 18:42:29.507817 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 5712
I20260807 18:42:29.507905 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/wal/instance
I20260807 18:42:29.508226 6697 external_mini_cluster.cc:1428] Setting key 32ea7930db14431a53aaa35c7c493cb0
I20260807 18:42:29.514879 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.507602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:59651 (local address 127.6.138.126:35645)
0807 18:42:29.507762 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.507767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.509047 (+ 1280us) server_negotiation.cc:407] Connection header received
0807 18:42:29.510459 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.510462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.510507 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.510580 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:29.511406 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.511947 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.513092 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.513304 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.513385 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.514487 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:29.514490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:29.514492 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:29.514503 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:29.514520 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:29.514527 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:29.514592 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.514722 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.514768 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:42:29.515762 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.508119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:35645 (local address 127.6.138.67:59651)
0807 18:42:29.508892 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:29.508945 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:29.510282 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:29.510710 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:29.510717 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:29.510796 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:42:29.511278 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.511291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.512061 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.512064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.512974 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.512983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.513437 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.513440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.513588 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.514248 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:42:29.514250 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:42:29.514291 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:42:29.514355 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:42:29.514602 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:42:29.514607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:42:29.514611 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 18:42:29.514699 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 18:42:29.514911 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":173,"threads_started":1}
I20260807 18:42:29.516912 5937 heartbeater.cc:347] Connected to a master server at 127.6.138.126:35645
I20260807 18:42:29.517119 5937 heartbeater.cc:464] Registering TS with master...
I20260807 18:42:29.517582 5937 heartbeater.cc:511] Master 127.6.138.126:35645 requested a full tablet report, sending...
I20260807 18:42:29.518553 5187 ts_manager.cc:194] Registered new tserver with Master: 9c37551485d1447a8851d6d73a0012f7 (127.6.138.67:45945)
I20260807 18:42:29.519110 5187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.138.67:59651
I20260807 18:42:29.521957 6697 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:42:29.527674 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.523905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.6.138.126:35645)
0807 18:42:29.524062 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.524069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.524376 (+ 307us) server_negotiation.cc:407] Connection header received
0807 18:42:29.524520 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.524526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.524587 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.524683 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:29.525050 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.525555 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.526366 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.526578 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.526663 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.527345 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:29.527348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:29.527350 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:29.527361 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:29.527379 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:29.527387 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:29.527443 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.527527 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.527574 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:42:29.530617 5187 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:42:29.533883 5187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:42:29.549091 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.542355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.67:45945 (local address 127.0.0.1:42274)
0807 18:42:29.542719 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:29.542777 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:29.542920 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:29.545142 (+ 2222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:29.545146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:29.545164 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:29.545258 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.545263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.546442 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.546445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.547548 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.547555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.548505 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.548508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.548592 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.548646 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:42:29.548782 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 18:42:29.548922 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":7,"spinlock_wait_cycles":3712,"thread_start_us":80,"threads_started":1}
I20260807 18:42:29.549093 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.543090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42274 (local address 127.6.138.67:45945)
0807 18:42:29.544494 (+ 1404us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.544506 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.544529 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:42:29.544607 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.544614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.544824 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.544976 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:29.545382 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.546304 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.547669 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.548202 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.548383 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.548594 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.548816 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.548921 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1296,"thread_start_us":176,"threads_started":1}
I20260807 18:42:29.549325 5956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.543428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.6.138.66:38403)
0807 18:42:29.543882 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.543897 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.543925 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:42:29.544029 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.544034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.544297 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.544475 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:29.544809 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.545788 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.547464 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.548284 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.548388 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.548588 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.548909 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.549006 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":150,"threads_started":1}
I20260807 18:42:29.549655 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.542455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:38403 (local address 127.0.0.1:42470)
0807 18:42:29.542719 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:29.542777 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:29.542920 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:29.544461 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:29.544467 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:29.544490 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:29.544647 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.544654 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.545925 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.545929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.547352 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.547363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.548393 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.548397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.548735 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.548756 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:42:29.548820 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 18:42:29.549526 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":827,"spinlock_wait_cycles":5632,"thread_start_us":139,"threads_started":1}
I20260807 18:42:29.550705 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.543472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.6.138.65:34761)
0807 18:42:29.544003 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.544015 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.544249 (+ 234us) server_negotiation.cc:407] Connection header received
0807 18:42:29.544418 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.544425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.544634 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.544793 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:29.546292 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.547246 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.548660 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.549828 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.549944 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.550122 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.550450 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.550551 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":168,"threads_started":1}
I20260807 18:42:29.551129 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.543131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34761 (local address 127.0.0.1:52822)
0807 18:42:29.544143 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:29.544154 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:29.544291 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:29.545915 (+ 1624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:29.545918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:29.545935 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:29.546007 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.546012 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.547363 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.547369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.548552 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.548563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.550183 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.550186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.550286 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.550311 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:42:29.550911 (+ 600us) client_negotiation.cc:240] Negotiation successful
0807 18:42:29.550989 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":24,"thread_start_us":98,"threads_started":1}
I20260807 18:42:29.552378 5819 tablet_service.cc:1467] Processing CreateTablet for tablet b44cf651bb8f40ac936ce62d9a7c33c3 (DEFAULT_TABLE table=test-table [id=dd69fe96ad7e40a4a19d778973576c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:29.552593 5603 tablet_service.cc:1467] Processing CreateTablet for tablet b44cf651bb8f40ac936ce62d9a7c33c3 (DEFAULT_TABLE table=test-table [id=dd69fe96ad7e40a4a19d778973576c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:29.553467 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44cf651bb8f40ac936ce62d9a7c33c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:29.553510 5819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44cf651bb8f40ac936ce62d9a7c33c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:29.553908 5402 tablet_service.cc:1467] Processing CreateTablet for tablet b44cf651bb8f40ac936ce62d9a7c33c3 (DEFAULT_TABLE table=test-table [id=dd69fe96ad7e40a4a19d778973576c74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:42:29.554843 5402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44cf651bb8f40ac936ce62d9a7c33c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:29.559971 5958 tablet_bootstrap.cc:492] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: Bootstrap starting.
I20260807 18:42:29.561607 5960 tablet_bootstrap.cc:492] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610: Bootstrap starting.
I20260807 18:42:29.561815 5959 tablet_bootstrap.cc:492] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: Bootstrap starting.
I20260807 18:42:29.562525 5958 tablet_bootstrap.cc:654] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:29.563417 5958 log.cc:824] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:29.563649 5959 tablet_bootstrap.cc:654] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:29.563766 5960 tablet_bootstrap.cc:654] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:29.564523 5959 log.cc:824] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:29.564658 5960 log.cc:824] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:29.565927 5958 tablet_bootstrap.cc:492] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: No bootstrap required, opened a new log
I20260807 18:42:29.566242 5958 ts_tablet_manager.cc:1397] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:42:29.566509 5960 tablet_bootstrap.cc:492] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610: No bootstrap required, opened a new log
I20260807 18:42:29.566605 5959 tablet_bootstrap.cc:492] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: No bootstrap required, opened a new log
I20260807 18:42:29.566716 5960 ts_tablet_manager.cc:1397] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:42:29.566813 5959 ts_tablet_manager.cc:1397] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:42:29.570075 5960 raft_consensus.cc:348] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } }
I20260807 18:42:29.570331 5960 raft_consensus.cc:374] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:29.570393 5960 raft_consensus.cc:729] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb198b323a4ee6b6c15ab7226f4610, State: Initialized, Role: FOLLOWER
I20260807 18:42:29.570663 5958 raft_consensus.cc:348] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } }
I20260807 18:42:29.570914 5958 raft_consensus.cc:374] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:29.570866 5960 consensus_queue.cc:260] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } }
I20260807 18:42:29.570977 5958 raft_consensus.cc:729] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abeb497278164496b5c7f813d5030d48, State: Initialized, Role: FOLLOWER
I20260807 18:42:29.570953 5959 raft_consensus.cc:348] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } }
I20260807 18:42:29.571187 5959 raft_consensus.cc:374] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:29.571259 5959 raft_consensus.cc:729] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c37551485d1447a8851d6d73a0012f7, State: Initialized, Role: FOLLOWER
I20260807 18:42:29.571457 5958 consensus_queue.cc:260] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } }
I20260807 18:42:29.571646 5959 consensus_queue.cc:260] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } }
I20260807 18:42:29.572048 5960 ts_tablet_manager.cc:1428] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 18:42:29.572216 5707 heartbeater.cc:503] Master 127.6.138.126:35645 was elected leader, sending a full tablet report...
I20260807 18:42:29.572395 5492 heartbeater.cc:503] Master 127.6.138.126:35645 was elected leader, sending a full tablet report...
I20260807 18:42:29.572471 5958 ts_tablet_manager.cc:1428] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:42:29.573050 5937 heartbeater.cc:503] Master 127.6.138.126:35645 was elected leader, sending a full tablet report...
I20260807 18:42:29.573601 5959 ts_tablet_manager.cc:1428] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 18:42:29.575865 5967 raft_consensus.cc:482] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:42:29.576016 5967 raft_consensus.cc:504] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } }
I20260807 18:42:29.577766 5967 leader_election.cc:288] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abeb497278164496b5c7f813d5030d48 (127.6.138.66:38403), 3afb198b323a4ee6b6c15ab7226f4610 (127.6.138.65:34761)
I20260807 18:42:29.581629 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.577399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.66:38403 (local address 127.6.138.67:46087)
0807 18:42:29.577544 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:29.577561 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:29.577641 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:29.578056 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:29.578059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:29.578079 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:29.578199 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.578214 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.579038 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.579044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.580362 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.580372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.581223 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.581226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.581335 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.581357 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:42:29.581417 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 18:42:29.581470 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 18:42:29.581727 5956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.577595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:46087 (local address 127.6.138.66:38403)
0807 18:42:29.577716 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.577720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.577735 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:42:29.577784 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.577788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.577852 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.577932 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:29.578326 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.578919 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.580474 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.581087 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.581228 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.581276 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.581458 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.581560 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:42:29.582286 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44cf651bb8f40ac936ce62d9a7c33c3" candidate_uuid: "9c37551485d1447a8851d6d73a0012f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abeb497278164496b5c7f813d5030d48" is_pre_election: true
I20260807 18:42:29.582592 5638 raft_consensus.cc:2454] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c37551485d1447a8851d6d73a0012f7 in term 0.
I20260807 18:42:29.582651 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.578030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.138.67:45569 (local address 127.6.138.65:34761)
0807 18:42:29.578131 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.578135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.578758 (+ 623us) server_negotiation.cc:407] Connection header received
0807 18:42:29.578895 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.578898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.578940 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.579065 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:42:29.579334 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.580079 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.581551 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.582097 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.582190 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.582221 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.582489 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.582566 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:42:29.582672 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.577963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34761 (local address 127.6.138.67:45569)
0807 18:42:29.578661 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:29.578673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:42:29.578791 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:42:29.579110 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:42:29.579112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:42:29.579129 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:42:29.579229 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.579233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.580192 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.580194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.581425 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:42:29.581437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.582241 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.582245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:42:29.582350 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.582373 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:42:29.582460 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 18:42:29.582528 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":126,"threads_started":1}
I20260807 18:42:29.583142 5746 leader_election.cc:302] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c37551485d1447a8851d6d73a0012f7, abeb497278164496b5c7f813d5030d48; no voters:
I20260807 18:42:29.583556 5967 raft_consensus.cc:2781] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:42:29.583654 5967 raft_consensus.cc:482] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:42:29.583714 5967 raft_consensus.cc:3037] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:29.583837 5426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44cf651bb8f40ac936ce62d9a7c33c3" candidate_uuid: "9c37551485d1447a8851d6d73a0012f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb198b323a4ee6b6c15ab7226f4610" is_pre_election: true
I20260807 18:42:29.584141 5426 raft_consensus.cc:2454] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c37551485d1447a8851d6d73a0012f7 in term 0.
I20260807 18:42:29.585026 5967 raft_consensus.cc:504] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } }
I20260807 18:42:29.585390 5967 leader_election.cc:288] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [CANDIDATE]: Term 1 election: Requested vote from peers abeb497278164496b5c7f813d5030d48 (127.6.138.66:38403), 3afb198b323a4ee6b6c15ab7226f4610 (127.6.138.65:34761)
I20260807 18:42:29.585722 5638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44cf651bb8f40ac936ce62d9a7c33c3" candidate_uuid: "9c37551485d1447a8851d6d73a0012f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abeb497278164496b5c7f813d5030d48"
I20260807 18:42:29.585788 5426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44cf651bb8f40ac936ce62d9a7c33c3" candidate_uuid: "9c37551485d1447a8851d6d73a0012f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3afb198b323a4ee6b6c15ab7226f4610"
I20260807 18:42:29.585867 5638 raft_consensus.cc:3037] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:29.585902 5426 raft_consensus.cc:3037] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:29.586982 5638 raft_consensus.cc:2454] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c37551485d1447a8851d6d73a0012f7 in term 1.
I20260807 18:42:29.587067 5426 raft_consensus.cc:2454] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c37551485d1447a8851d6d73a0012f7 in term 1.
I20260807 18:42:29.587303 5746 leader_election.cc:302] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c37551485d1447a8851d6d73a0012f7, abeb497278164496b5c7f813d5030d48; no voters:
I20260807 18:42:29.587509 5967 raft_consensus.cc:2781] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:29.587751 5967 raft_consensus.cc:686] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 1 LEADER]: Becoming Leader. State: Replica: 9c37551485d1447a8851d6d73a0012f7, State: Running, Role: LEADER
I20260807 18:42:29.588078 5967 consensus_queue.cc:237] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } }
I20260807 18:42:29.591264 5187 catalog_manager.cc:5495] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c37551485d1447a8851d6d73a0012f7 (127.6.138.67). New cstate: current_term: 1 leader_uuid: "9c37551485d1447a8851d6d73a0012f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c37551485d1447a8851d6d73a0012f7" member_type: VOTER last_known_addr { host: "127.6.138.67" port: 45945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 } health_report { overall_health: UNKNOWN } } }
I20260807 18:42:29.601691 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:29.598092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.6.138.67:45945)
0807 18:42:29.598209 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:29.598215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:29.598274 (+ 59us) server_negotiation.cc:407] Connection header received
0807 18:42:29.598464 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:29.598466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:29.598518 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:29.598632 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:42:29.598886 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.599389 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.600241 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:29.600493 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:29.600610 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:29.600921 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:42:29.601304 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:42:29.601439 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:29.601500 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 18:42:29.601554 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 18:42:29.607627 5493 tablet.cc:2369] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:42:29.607777 5638 raft_consensus.cc:1261] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [term 1 FOLLOWER]: Refusing update from remote peer 9c37551485d1447a8851d6d73a0012f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:29.607777 5426 raft_consensus.cc:1261] T b44cf651bb8f40ac936ce62d9a7c33c3 P 3afb198b323a4ee6b6c15ab7226f4610 [term 1 FOLLOWER]: Refusing update from remote peer 9c37551485d1447a8851d6d73a0012f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:42:29.608568 5967 consensus_queue.cc:1035] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3afb198b323a4ee6b6c15ab7226f4610" member_type: VOTER last_known_addr { host: "127.6.138.65" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:29.608778 5970 consensus_queue.cc:1035] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abeb497278164496b5c7f813d5030d48" member_type: VOTER last_known_addr { host: "127.6.138.66" port: 38403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:42:29.626242 5187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:29.626442 5187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:42:29.629046 5187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d60e20fe7cd4b22b5d6ff3da326ed9c: Sending DeleteTablet for 3 replicas of tablet b44cf651bb8f40ac936ce62d9a7c33c3
I20260807 18:42:29.629884 5819 tablet_service.cc:1514] Processing DeleteTablet for tablet b44cf651bb8f40ac936ce62d9a7c33c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:29 UTC) from {username='slave'} at 127.0.0.1:42274
I20260807 18:42:29.629913 5402 tablet_service.cc:1514] Processing DeleteTablet for tablet b44cf651bb8f40ac936ce62d9a7c33c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:29 UTC) from {username='slave'} at 127.0.0.1:52822
I20260807 18:42:29.629968 5603 tablet_service.cc:1514] Processing DeleteTablet for tablet b44cf651bb8f40ac936ce62d9a7c33c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:42:29 UTC) from {username='slave'} at 127.0.0.1:42470
I20260807 18:42:29.630502 5990 tablet_replica.cc:330] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: stopping tablet replica
I20260807 18:42:29.630810 5988 tablet_replica.cc:330] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: stopping tablet replica
I20260807 18:42:29.630832 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 5287
I20260807 18:42:29.630848 5990 raft_consensus.cc:2229] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:42:29.631189 5990 raft_consensus.cc:2258] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:29.637260 5988 raft_consensus.cc:2229] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:42:29.637730 5988 raft_consensus.cc:2258] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:42:29.640064 5988 ts_tablet_manager.cc:1905] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:29.642488 5988 ts_tablet_manager.cc:1918] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:29.642621 5988 log.cc:1196] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-2/wal/wals/b44cf651bb8f40ac936ce62d9a7c33c3
I20260807 18:42:29.643020 5988 ts_tablet_manager.cc:1939] T b44cf651bb8f40ac936ce62d9a7c33c3 P 9c37551485d1447a8851d6d73a0012f7: Deleting consensus metadata
I20260807 18:42:29.644411 5168 catalog_manager.cc:4841] TS 9c37551485d1447a8851d6d73a0012f7 (127.6.138.67:45945): tablet b44cf651bb8f40ac936ce62d9a7c33c3 (table test-table [id=dd69fe96ad7e40a4a19d778973576c74]) successfully deleted
I20260807 18:42:29.644840 5990 ts_tablet_manager.cc:1905] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:42:29.647045 5990 ts_tablet_manager.cc:1918] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:42:29.647173 5990 log.cc:1196] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: Deleting WAL directory at /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127881318083-6697-0/minicluster-data/ts-1/wal/wals/b44cf651bb8f40ac936ce62d9a7c33c3
I20260807 18:42:29.647473 5990 ts_tablet_manager.cc:1939] T b44cf651bb8f40ac936ce62d9a7c33c3 P abeb497278164496b5c7f813d5030d48: Deleting consensus metadata
W20260807 18:42:29.648604 5167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.138.65:34761 (error 108)
I20260807 18:42:29.649152 5167 catalog_manager.cc:4841] TS abeb497278164496b5c7f813d5030d48 (127.6.138.66:38403): tablet b44cf651bb8f40ac936ce62d9a7c33c3 (table test-table [id=dd69fe96ad7e40a4a19d778973576c74]) successfully deleted
W20260807 18:42:29.649746 5953 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:42:29.649466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.65:34761 (local address 127.0.0.1:52828)
0807 18:42:29.649601 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 18:42:29.649666 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.65:34761: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:42:29.649940 5167 catalog_manager.cc:4584] TS 3afb198b323a4ee6b6c15ab7226f4610 (127.6.138.65:34761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b44cf651bb8f40ac936ce62d9a7c33c3: Network error: Client connection negotiation failed: client connection to 127.6.138.65:34761: connect: Connection reset by peer (error 104)
I20260807 18:42:29.650009 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 5497
I20260807 18:42:29.662889 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 5712
I20260807 18:42:29.682504 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 5145
2026-08-07T18:42:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1124 ms)
[----------] 6 tests from EncryptionPolicyTest (11606 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:29 dist-test-slave-2d19 krb5kdc[5995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:29 dist-test-slave-2d19 krb5kdc[5995](info): set up 2 sockets
Aug 07 18:42:29 dist-test-slave-2d19 krb5kdc[5995](info): commencing operation
krb5kdc: starting...
W20260807 18:42:30.938299 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:31 dist-test-slave-2d19 krb5kdc[5995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:31.072423 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:31.195182 6022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:31.195498 6022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:31.195578 6022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:31.199572 6022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:31.199673 6022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:31.199712 6022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:31.199750 6022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:31.199790 6022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:31.205137 6022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45629
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:31.206254 6022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:31.207386 6022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:31.216632 6040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:31.216809 6038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:31.216677 6034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:31.216969 6022 server_base.cc:1034] running on GCE node
I20260807 18:42:31.217609 6022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:31.218750 6022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:31.219941 6022 hybrid_clock.cc:648] HybridClock initialized: now 1786128151219930 us; error 34 us; skew 500 ppm
Aug 07 18:42:31 dist-test-slave-2d19 krb5kdc[5995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:31.222774 6022 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:31.223711 6022 webserver.cc:460] Webserver started at http://127.0.0.1:43069/ using document root <none> and password file <none>
I20260807 18:42:31.224350 6022 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:31.224443 6022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:31.224689 6022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:31.226737 6022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "c26845f3ced6435eb2882f884c9dd736"
format_stamp: "Formatted at 2026-08-07 18:42:31 on dist-test-slave-2d19"
server_key: "95da19b1e8ac05a5821e96827c5f433a"
server_key_iv: "8534ca05a4c9b1537ce69d37946b3fc7"
server_key_version: "encryptionkey@0"
I20260807 18:42:31.227319 6022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "c26845f3ced6435eb2882f884c9dd736"
format_stamp: "Formatted at 2026-08-07 18:42:31 on dist-test-slave-2d19"
server_key: "95da19b1e8ac05a5821e96827c5f433a"
server_key_iv: "8534ca05a4c9b1537ce69d37946b3fc7"
server_key_version: "encryptionkey@0"
I20260807 18:42:31.231698 6022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:42:31.235184 6049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:31.236827 6022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:42:31.236963 6022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "c26845f3ced6435eb2882f884c9dd736"
format_stamp: "Formatted at 2026-08-07 18:42:31 on dist-test-slave-2d19"
server_key: "95da19b1e8ac05a5821e96827c5f433a"
server_key_iv: "8534ca05a4c9b1537ce69d37946b3fc7"
server_key_version: "encryptionkey@0"
I20260807 18:42:31.237095 6022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:31.271147 6022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:31.274857 6022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:31.275045 6022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:31.282605 6022 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45629
I20260807 18:42:31.282680 6143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45629 every 8 connection(s)
I20260807 18:42:31.283649 6022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:31.286895 6144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:31.290761 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 6022
I20260807 18:42:31.290920 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:31.291222 6697 external_mini_cluster.cc:1428] Setting key bff0339bc2862f8fa834bca856756910
I20260807 18:42:31.293736 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736: Bootstrap starting.
I20260807 18:42:31.296226 6144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:31.297102 6144 log.cc:824] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:31.299276 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736: No bootstrap required, opened a new log
Aug 07 18:42:31 dist-test-slave-2d19 krb5kdc[5995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:42:31.303153 6144 raft_consensus.cc:348] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26845f3ced6435eb2882f884c9dd736" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } }
I20260807 18:42:31.303452 6144 raft_consensus.cc:374] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:31.303540 6144 raft_consensus.cc:729] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c26845f3ced6435eb2882f884c9dd736, State: Initialized, Role: FOLLOWER
I20260807 18:42:31.304064 6144 consensus_queue.cc:260] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26845f3ced6435eb2882f884c9dd736" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } }
I20260807 18:42:31.304239 6144 raft_consensus.cc:388] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:31.304339 6144 raft_consensus.cc:482] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:31.304462 6144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:31.305478 6144 raft_consensus.cc:504] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26845f3ced6435eb2882f884c9dd736" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } }
I20260807 18:42:31.305850 6144 leader_election.cc:302] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c26845f3ced6435eb2882f884c9dd736; no voters:
I20260807 18:42:31.306242 6144 leader_election.cc:288] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:31.306417 6150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:31.306707 6150 raft_consensus.cc:686] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [term 1 LEADER]: Becoming Leader. State: Replica: c26845f3ced6435eb2882f884c9dd736, State: Running, Role: LEADER
I20260807 18:42:31.307087 6150 consensus_queue.cc:237] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26845f3ced6435eb2882f884c9dd736" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } }
I20260807 18:42:31.307868 6144 sys_catalog.cc:564] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:31.308307 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:31.292814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35915 (local address 127.0.0.1:45629)
0807 18:42:31.293343 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:31.293364 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:31.293401 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:42:31.294133 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:31.294153 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:31.294510 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:31.294876 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:31.295752 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:31.296893 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:31.297859 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:31.298345 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:31.298532 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:31.301284 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:31.301290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:31.301303 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:31.301331 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:31.303978 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:31.304801 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:31.304808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:31.304813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:31.304950 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:31.305388 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:31.305392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:31.305393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:31.305929 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:31.306175 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:31.306902 (+ 727us) server_negotiation.cc:299] Negotiation successful
0807 18:42:31.307210 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":170,"threads_started":1}
I20260807 18:42:31.308459 6153 sys_catalog.cc:455] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c26845f3ced6435eb2882f884c9dd736. Latest consensus state: current_term: 1 leader_uuid: "c26845f3ced6435eb2882f884c9dd736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26845f3ced6435eb2882f884c9dd736" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } } }
I20260807 18:42:31.308627 6153 sys_catalog.cc:458] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:31.309170 6161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:31.309212 6151 sys_catalog.cc:455] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c26845f3ced6435eb2882f884c9dd736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c26845f3ced6435eb2882f884c9dd736" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45629 } } }
I20260807 18:42:31.309294 6151 sys_catalog.cc:458] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:31.312041 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:31.312884 6161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:31.319600 6161 catalog_manager.cc:1347] Generated new cluster ID: 19a30c9ad0ed4416ab0945f49234fb1a
I20260807 18:42:31.319667 6161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:31.325454 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:31.315761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42833 (local address 127.0.0.1:45629)
0807 18:42:31.315869 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:31.315874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:31.316088 (+ 214us) server_negotiation.cc:407] Connection header received
0807 18:42:31.316208 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:31.316211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:31.316300 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:31.316388 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:31.317264 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:31.317954 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:31.319153 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:31.319512 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:31.319591 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:31.321591 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:31.321596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:31.321599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:31.321626 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:31.323434 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:31.324010 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:31.324014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:31.324017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:31.324105 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:31.324511 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:31.324515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:31.324517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:31.324737 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:31.324905 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:31.325187 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:42:31.325289 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 18:42:31.331848 6161 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:31.333072 6161 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:31.338748 6161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c26845f3ced6435eb2882f884c9dd736: Generated new TSK 0
I20260807 18:42:31.339533 6161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:31.343595 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 6022
W20260807 18:42:31.352888 5993 connection.cc:535] client connection to 127.0.0.1:45629 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45629): Connection reset by peer (error 104)
2026-08-07T18:42:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1656 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:31 dist-test-slave-2d19 krb5kdc[6193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:31 dist-test-slave-2d19 krb5kdc[6193](info): set up 2 sockets
Aug 07 18:42:31 dist-test-slave-2d19 krb5kdc[6193](info): commencing operation
krb5kdc: starting...
W20260807 18:42:32.604440 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:32 dist-test-slave-2d19 krb5kdc[6193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:32.739259 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:32.855602 6213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:32.855947 6213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:32.856031 6213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:32.860114 6213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:32.860222 6213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:32.860263 6213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:32.860323 6213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:32.860360 6213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:32.865808 6213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:32.866955 6213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:32.868022 6213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:32.876838 6228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:32.876796 6224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:32.876796 6223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:32.877054 6213 server_base.cc:1034] running on GCE node
I20260807 18:42:32.877678 6213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:32.878823 6213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:32.880020 6213 hybrid_clock.cc:648] HybridClock initialized: now 1786128152879991 us; error 43 us; skew 500 ppm
Aug 07 18:42:32 dist-test-slave-2d19 krb5kdc[6193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128152, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:32.882947 6213 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:32.883836 6213 webserver.cc:460] Webserver started at http://127.0.0.1:43961/ using document root <none> and password file <none>
I20260807 18:42:32.884493 6213 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:32.884559 6213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:32.884824 6213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:32.886950 6213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "945fcdf6e4564ec390496d0e301ca4fe"
format_stamp: "Formatted at 2026-08-07 18:42:32 on dist-test-slave-2d19"
server_key: "4ad4b153f277dae7f7064dfb8dd3f20d"
server_key_iv: "b7e6a2757aa19d446f0ee5c3af0b87a3"
server_key_version: "encryptionkey@0"
I20260807 18:42:32.887531 6213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "945fcdf6e4564ec390496d0e301ca4fe"
format_stamp: "Formatted at 2026-08-07 18:42:32 on dist-test-slave-2d19"
server_key: "4ad4b153f277dae7f7064dfb8dd3f20d"
server_key_iv: "b7e6a2757aa19d446f0ee5c3af0b87a3"
server_key_version: "encryptionkey@0"
I20260807 18:42:32.891363 6213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:42:32.893910 6239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:32.895138 6213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:42:32.895272 6213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "945fcdf6e4564ec390496d0e301ca4fe"
format_stamp: "Formatted at 2026-08-07 18:42:32 on dist-test-slave-2d19"
server_key: "4ad4b153f277dae7f7064dfb8dd3f20d"
server_key_iv: "b7e6a2757aa19d446f0ee5c3af0b87a3"
server_key_version: "encryptionkey@0"
I20260807 18:42:32.895397 6213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:32.909132 6213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:32.912585 6213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:32.912787 6213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:32.919863 6213 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41161
I20260807 18:42:32.919925 6326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41161 every 8 connection(s)
I20260807 18:42:32.920905 6213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:32.923763 6328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:32.926168 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 6213
I20260807 18:42:32.926319 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:32.926661 6697 external_mini_cluster.cc:1428] Setting key 60fe9b79d85df0cddd2c67d1a7f9d827
I20260807 18:42:32.929991 6328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe: Bootstrap starting.
I20260807 18:42:32.932207 6328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:32.932971 6328 log.cc:824] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:32.935169 6328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe: No bootstrap required, opened a new log
I20260807 18:42:32.935921 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:32.928064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59729 (local address 127.0.0.1:41161)
0807 18:42:32.928580 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:32.928595 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:32.928633 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:42:32.929277 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:32.929293 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:32.929640 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:32.929995 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:32.930420 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:32.931387 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:32.932387 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:32.932858 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:32.933040 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:32.934043 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:32.934049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:32.934062 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:32.934470 (+ 408us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:32.934527 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:32.934557 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:32.934681 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:32.934985 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 18:42:32.935152 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":168,"threads_started":1}
I20260807 18:42:32.939270 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:32.939147 6328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945fcdf6e4564ec390496d0e301ca4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260807 18:42:32.939431 6328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:32.939484 6328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945fcdf6e4564ec390496d0e301ca4fe, State: Initialized, Role: FOLLOWER
I20260807 18:42:32.939945 6328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945fcdf6e4564ec390496d0e301ca4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260807 18:42:32.940121 6328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:32.940215 6328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:32.940340 6328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:32.941566 6328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945fcdf6e4564ec390496d0e301ca4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260807 18:42:32.941949 6328 leader_election.cc:302] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 945fcdf6e4564ec390496d0e301ca4fe; no voters:
I20260807 18:42:32.942346 6328 leader_election.cc:288] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:32.942468 6344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:32.942718 6344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [term 1 LEADER]: Becoming Leader. State: Replica: 945fcdf6e4564ec390496d0e301ca4fe, State: Running, Role: LEADER
I20260807 18:42:32.943023 6344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945fcdf6e4564ec390496d0e301ca4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260807 18:42:32.943660 6328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:32.944804 6348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 945fcdf6e4564ec390496d0e301ca4fe. Latest consensus state: current_term: 1 leader_uuid: "945fcdf6e4564ec390496d0e301ca4fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945fcdf6e4564ec390496d0e301ca4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } } }
I20260807 18:42:32.944815 6346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "945fcdf6e4564ec390496d0e301ca4fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "945fcdf6e4564ec390496d0e301ca4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } } }
I20260807 18:42:32.944950 6346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:32.944950 6348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:32.945358 6357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:32.946801 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:32.941599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33257 (local address 127.0.0.1:41161)
0807 18:42:32.941728 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:32.941733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:32.941981 (+ 248us) server_negotiation.cc:407] Connection header received
0807 18:42:32.942121 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:32.942127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:32.942176 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:32.942239 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:32.942633 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:32.943378 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:32.944651 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:32.944999 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:32.945079 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:32.946278 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:32.946281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:32.946284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:32.946297 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:32.946330 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:32.946341 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:32.946450 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:32.946636 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 18:42:32.946674 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:42:32.948956 6357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:32.955003 6357 catalog_manager.cc:1347] Generated new cluster ID: 313b062edbc642d5b7cbebe0437616b1
I20260807 18:42:32.955087 6357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:32.968120 6357 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:32.969055 6357 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:32.978188 6357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 945fcdf6e4564ec390496d0e301ca4fe: Generated new TSK 0
I20260807 18:42:32.978989 6357 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:32.992825 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 6213
W20260807 18:42:33.005053 6188 connection.cc:535] client connection to 127.0.0.1:41161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41161): Connection reset by peer (error 104)
2026-08-07T18:42:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1651 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:33 dist-test-slave-2d19 krb5kdc[6380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:33 dist-test-slave-2d19 krb5kdc[6380](info): set up 2 sockets
Aug 07 18:42:33 dist-test-slave-2d19 krb5kdc[6380](info): commencing operation
krb5kdc: starting...
W20260807 18:42:34.274528 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:34 dist-test-slave-2d19 krb5kdc[6380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:34.415256 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45913
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:34.531129 6402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:34.531452 6402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:34.531529 6402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:34.535589 6402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:34.535701 6402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:34.535741 6402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:34.535780 6402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:34.535811 6402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:34.541185 6402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45913
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:34.542317 6402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:34.543416 6402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:34.552381 6413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:34.552361 6408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:34.552361 6409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:34.552425 6402 server_base.cc:1034] running on GCE node
I20260807 18:42:34.553191 6402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:34.554350 6402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:34.555570 6402 hybrid_clock.cc:648] HybridClock initialized: now 1786128154555558 us; error 28 us; skew 500 ppm
Aug 07 18:42:34 dist-test-slave-2d19 krb5kdc[6380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128154, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:34.562795 6402 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:34.563774 6402 webserver.cc:460] Webserver started at http://127.0.0.1:45209/ using document root <none> and password file <none>
I20260807 18:42:34.564373 6402 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:34.564442 6402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:34.564639 6402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:34.566709 6402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "311485661ecc4ba7b6942e80e0f58f32"
format_stamp: "Formatted at 2026-08-07 18:42:34 on dist-test-slave-2d19"
server_key: "3bf73fd3eb2c97843f890566163556ef"
server_key_iv: "88d9d240bba3b53dbb310763fa072532"
server_key_version: "encryptionkey@0"
I20260807 18:42:34.567230 6402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "311485661ecc4ba7b6942e80e0f58f32"
format_stamp: "Formatted at 2026-08-07 18:42:34 on dist-test-slave-2d19"
server_key: "3bf73fd3eb2c97843f890566163556ef"
server_key_iv: "88d9d240bba3b53dbb310763fa072532"
server_key_version: "encryptionkey@0"
I20260807 18:42:34.571079 6402 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:42:34.573596 6424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:34.574841 6402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:42:34.574944 6402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "311485661ecc4ba7b6942e80e0f58f32"
format_stamp: "Formatted at 2026-08-07 18:42:34 on dist-test-slave-2d19"
server_key: "3bf73fd3eb2c97843f890566163556ef"
server_key_iv: "88d9d240bba3b53dbb310763fa072532"
server_key_version: "encryptionkey@0"
I20260807 18:42:34.575037 6402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:34.579658 6402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:34.582449 6402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:34.582614 6402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:34.589887 6402 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45913
I20260807 18:42:34.589915 6516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45913 every 8 connection(s)
I20260807 18:42:34.590833 6402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:34.592716 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 6402
I20260807 18:42:34.592870 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:34.593192 6697 external_mini_cluster.cc:1428] Setting key 11dd15f9c106bdae15a32f4c3c1f7cc5
I20260807 18:42:34.593954 6517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:34.598510 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:34.594643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37983 (local address 127.0.0.1:45913)
0807 18:42:34.595117 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:34.595132 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:34.595169 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:42:34.595842 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:34.595858 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:34.596145 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:34.596537 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:34.597353 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:34.597358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:34.597370 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:34.597380 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:34.597433 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:34.597452 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:34.597608 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:34.597681 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 18:42:34.597840 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":150,"threads_started":1}
I20260807 18:42:34.600667 6517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32: Bootstrap starting.
I20260807 18:42:34.601378 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:34.602974 6517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:34.603852 6517 log.cc:824] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:34.605165 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:34.603368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57103 (local address 127.0.0.1:45913)
0807 18:42:34.603486 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:34.603490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:34.603696 (+ 206us) server_negotiation.cc:407] Connection header received
0807 18:42:34.603830 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:34.603832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:34.603864 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:34.603954 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:34.604730 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:34.604734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:34.604738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:34.604750 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:34.604776 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:34.604790 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:34.604898 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:34.605011 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 18:42:34.605050 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:42:34.606277 6517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32: No bootstrap required, opened a new log
I20260807 18:42:34.609900 6517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311485661ecc4ba7b6942e80e0f58f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } }
I20260807 18:42:34.610118 6517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:34.610198 6517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 311485661ecc4ba7b6942e80e0f58f32, State: Initialized, Role: FOLLOWER
I20260807 18:42:34.610672 6517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311485661ecc4ba7b6942e80e0f58f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } }
I20260807 18:42:34.610816 6517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:34.610906 6517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:34.611027 6517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:34.611971 6517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311485661ecc4ba7b6942e80e0f58f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } }
I20260807 18:42:34.612360 6517 leader_election.cc:302] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 311485661ecc4ba7b6942e80e0f58f32; no voters:
I20260807 18:42:34.612716 6517 leader_election.cc:288] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:34.612838 6531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:34.613134 6531 raft_consensus.cc:686] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [term 1 LEADER]: Becoming Leader. State: Replica: 311485661ecc4ba7b6942e80e0f58f32, State: Running, Role: LEADER
I20260807 18:42:34.613511 6531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311485661ecc4ba7b6942e80e0f58f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } }
I20260807 18:42:34.613978 6517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:34.615198 6533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "311485661ecc4ba7b6942e80e0f58f32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311485661ecc4ba7b6942e80e0f58f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } } }
I20260807 18:42:34.615353 6533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:34.615669 6538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 311485661ecc4ba7b6942e80e0f58f32. Latest consensus state: current_term: 1 leader_uuid: "311485661ecc4ba7b6942e80e0f58f32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311485661ecc4ba7b6942e80e0f58f32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45913 } } }
I20260807 18:42:34.615751 6551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:34.615777 6538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:34.618976 6551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:34.624914 6551 catalog_manager.cc:1347] Generated new cluster ID: 4f5ce719b4f44d47b3746766be09842a
I20260807 18:42:34.625020 6551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:34.634799 6551 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:34.635191 6551 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:34.643159 6551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 311485661ecc4ba7b6942e80e0f58f32: Generated new TSK 0
I20260807 18:42:34.643851 6551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:34.648007 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 6402
2026-08-07T18:42:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1659 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:34 dist-test-slave-2d19 krb5kdc[6571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:34 dist-test-slave-2d19 krb5kdc[6571](info): set up 2 sockets
Aug 07 18:42:34 dist-test-slave-2d19 krb5kdc[6571](info): commencing operation
krb5kdc: starting...
W20260807 18:42:35.943500 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:36 dist-test-slave-2d19 krb5kdc[6571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:36.079053 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:36.194705 6591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:36.195149 6591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:36.195235 6591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:36.199596 6591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:36.199659 6591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:36.199679 6591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:36.199697 6591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:36.199753 6591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:36.205214 6591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:36.206297 6591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:36.207338 6591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:36.215936 6603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:36.215931 6600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:36.215936 6598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:36.216135 6591 server_base.cc:1034] running on GCE node
I20260807 18:42:36.216837 6591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:36.218004 6591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:36.219205 6591 hybrid_clock.cc:648] HybridClock initialized: now 1786128156219181 us; error 43 us; skew 500 ppm
Aug 07 18:42:36 dist-test-slave-2d19 krb5kdc[6571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:36.222121 6591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:36.223001 6591 webserver.cc:460] Webserver started at http://127.0.0.1:37525/ using document root <none> and password file <none>
I20260807 18:42:36.223577 6591 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:36.223675 6591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:36.223920 6591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:36.226114 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "e5b55dbe67d54f94a0836644e385958b"
format_stamp: "Formatted at 2026-08-07 18:42:36 on dist-test-slave-2d19"
server_key: "af1053af777a396e824b7d62d7ddc0c6"
server_key_iv: "1b6a216f0e3b3f4e98f034857a384933"
server_key_version: "encryptionkey@0"
I20260807 18:42:36.226724 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "e5b55dbe67d54f94a0836644e385958b"
format_stamp: "Formatted at 2026-08-07 18:42:36 on dist-test-slave-2d19"
server_key: "af1053af777a396e824b7d62d7ddc0c6"
server_key_iv: "1b6a216f0e3b3f4e98f034857a384933"
server_key_version: "encryptionkey@0"
I20260807 18:42:36.230584 6591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:42:36.233244 6614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:36.234455 6591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:36.234598 6591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "e5b55dbe67d54f94a0836644e385958b"
format_stamp: "Formatted at 2026-08-07 18:42:36 on dist-test-slave-2d19"
server_key: "af1053af777a396e824b7d62d7ddc0c6"
server_key_iv: "1b6a216f0e3b3f4e98f034857a384933"
server_key_version: "encryptionkey@0"
I20260807 18:42:36.234716 6591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:36.248749 6591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:36.251843 6591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:36.252100 6591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:36.259410 6591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37811
I20260807 18:42:36.259419 6717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37811 every 8 connection(s)
I20260807 18:42:36.260443 6591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:36.263214 6718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:36.266422 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 6591
I20260807 18:42:36.266595 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:36.266875 6697 external_mini_cluster.cc:1428] Setting key 853a79855d501344a8615748fdf7eaec
I20260807 18:42:36.269371 6718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b: Bootstrap starting.
I20260807 18:42:36.271940 6718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:36.272927 6718 log.cc:824] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:36.275377 6718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b: No bootstrap required, opened a new log
Aug 07 18:42:36 dist-test-slave-2d19 krb5kdc[6571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:42:36.278497 6718 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b55dbe67d54f94a0836644e385958b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } }
I20260807 18:42:36.278764 6718 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:36.278826 6718 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b55dbe67d54f94a0836644e385958b, State: Initialized, Role: FOLLOWER
I20260807 18:42:36.279263 6718 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b55dbe67d54f94a0836644e385958b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } }
I20260807 18:42:36.279423 6718 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:36.279508 6718 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:36.279635 6718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:36.280699 6718 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b55dbe67d54f94a0836644e385958b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } }
I20260807 18:42:36.281051 6718 leader_election.cc:302] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5b55dbe67d54f94a0836644e385958b; no voters:
I20260807 18:42:36.281405 6718 leader_election.cc:288] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:36.281492 6724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:36.281735 6724 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [term 1 LEADER]: Becoming Leader. State: Replica: e5b55dbe67d54f94a0836644e385958b, State: Running, Role: LEADER
I20260807 18:42:36.282104 6724 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b55dbe67d54f94a0836644e385958b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } }
I20260807 18:42:36.282624 6722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:36.268223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60599 (local address 127.0.0.1:37811)
0807 18:42:36.268695 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:36.268712 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:36.268749 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:42:36.269362 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:36.269378 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:36.269764 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:36.270054 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:36.270997 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:36.272031 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:36.273262 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:36.273719 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:36.273837 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:36.276745 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:36.276751 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:36.276765 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:36.276791 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:36.279340 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:36.279882 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:36.279888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:36.279892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:36.280004 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:36.280351 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:36.280355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:36.280356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:36.280755 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:36.280972 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:36.281283 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 18:42:36.281614 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":129,"threads_started":1}
I20260807 18:42:36.282800 6718 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:36.283313 6724 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5b55dbe67d54f94a0836644e385958b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b55dbe67d54f94a0836644e385958b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } } }
I20260807 18:42:36.283488 6724 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:36.283747 6725 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5b55dbe67d54f94a0836644e385958b. Latest consensus state: current_term: 1 leader_uuid: "e5b55dbe67d54f94a0836644e385958b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b55dbe67d54f94a0836644e385958b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } } }
I20260807 18:42:36.284005 6725 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:36.284965 6732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:36.286163 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:36.288887 6732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:36.295101 6732 catalog_manager.cc:1347] Generated new cluster ID: 3930a4a524c14d749fe1921b7e5ad850
I20260807 18:42:36.295167 6732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:36.299481 6722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:36.288369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35085 (local address 127.0.0.1:37811)
0807 18:42:36.288506 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:36.288511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:36.291140 (+ 2629us) server_negotiation.cc:407] Connection header received
0807 18:42:36.291250 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:36.291253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:36.291301 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:36.291389 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:36.292329 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:36.293024 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:36.293968 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:36.294209 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:36.294320 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:36.295962 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:36.295966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:36.295969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:36.295998 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:36.297741 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:36.298215 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:36.298219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:36.298221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:36.298317 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:36.298598 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:36.298602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:36.298604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:36.298826 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:36.298962 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:36.299194 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 18:42:36.299319 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:42:36.325063 6732 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:36.326051 6732 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:36.333042 6732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5b55dbe67d54f94a0836644e385958b: Generated new TSK 0
I20260807 18:42:36.333758 6732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:36.340830 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 6591
2026-08-07T18:42:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1693 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:36 dist-test-slave-2d19 krb5kdc[6768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:36 dist-test-slave-2d19 krb5kdc[6768](info): set up 2 sockets
Aug 07 18:42:36 dist-test-slave-2d19 krb5kdc[6768](info): commencing operation
krb5kdc: starting...
W20260807 18:42:37.621183 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:37 dist-test-slave-2d19 krb5kdc[6768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:37.756487 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:37.876178 6791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:37.876591 6791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:37.876686 6791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:37.880802 6791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:37.880898 6791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:37.880945 6791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:37.881007 6791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:37.881045 6791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:37.886485 6791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:37.887617 6791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:37.888741 6791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:37.897706 6804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:37.897710 6801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:37.897895 6791 server_base.cc:1034] running on GCE node
W20260807 18:42:37.898026 6808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:37.898628 6791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:37.899740 6791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:37.900938 6791 hybrid_clock.cc:648] HybridClock initialized: now 1786128157900915 us; error 45 us; skew 500 ppm
Aug 07 18:42:37 dist-test-slave-2d19 krb5kdc[6768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:37.904100 6791 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:37.905141 6791 webserver.cc:460] Webserver started at http://127.0.0.1:43279/ using document root <none> and password file <none>
I20260807 18:42:37.905730 6791 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:37.905826 6791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:37.906083 6791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:37.908169 6791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "a9eeeeba7f4140b4a832937b0fabd697"
format_stamp: "Formatted at 2026-08-07 18:42:37 on dist-test-slave-2d19"
server_key: "139234d009bfdf74bb401cb88b136802"
server_key_iv: "0ec33136bb7a5a0fd56790d750cf0278"
server_key_version: "encryptionkey@0"
I20260807 18:42:37.908785 6791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "a9eeeeba7f4140b4a832937b0fabd697"
format_stamp: "Formatted at 2026-08-07 18:42:37 on dist-test-slave-2d19"
server_key: "139234d009bfdf74bb401cb88b136802"
server_key_iv: "0ec33136bb7a5a0fd56790d750cf0278"
server_key_version: "encryptionkey@0"
I20260807 18:42:37.912590 6791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:42:37.915140 6821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:37.916383 6791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:37.916518 6791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "a9eeeeba7f4140b4a832937b0fabd697"
format_stamp: "Formatted at 2026-08-07 18:42:37 on dist-test-slave-2d19"
server_key: "139234d009bfdf74bb401cb88b136802"
server_key_iv: "0ec33136bb7a5a0fd56790d750cf0278"
server_key_version: "encryptionkey@0"
I20260807 18:42:37.916632 6791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:37.935983 6791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:37.939309 6791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:37.939532 6791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:37.947103 6791 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37333
I20260807 18:42:37.947119 6911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37333 every 8 connection(s)
I20260807 18:42:37.948463 6791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:37.951301 6912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:37.954456 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 6791
I20260807 18:42:37.954617 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:37.954907 6697 external_mini_cluster.cc:1428] Setting key 39b81efa2395f55e916a3692a1394228
I20260807 18:42:37.957763 6912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697: Bootstrap starting.
I20260807 18:42:37.960566 6912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:37.961385 6912 log.cc:824] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:37.963459 6912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697: No bootstrap required, opened a new log
I20260807 18:42:37.964946 6915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:37.956480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58961 (local address 127.0.0.1:37333)
0807 18:42:37.956953 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:37.956973 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:37.957018 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:42:37.957869 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:37.957896 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:37.958354 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:37.958730 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:37.959197 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:37.960678 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:37.961589 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:37.962072 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:37.962244 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:37.963165 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:37.963170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:37.963184 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:37.963599 (+ 415us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:37.963670 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:37.963694 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:37.963800 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:37.963954 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 18:42:37.964174 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":127,"threads_started":1}
I20260807 18:42:37.967681 6912 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9eeeeba7f4140b4a832937b0fabd697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37333 } }
I20260807 18:42:37.967989 6912 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:37.968036 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:37.968081 6912 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9eeeeba7f4140b4a832937b0fabd697, State: Initialized, Role: FOLLOWER
I20260807 18:42:37.968544 6912 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9eeeeba7f4140b4a832937b0fabd697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37333 } }
I20260807 18:42:37.968676 6912 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:37.968761 6912 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:37.968860 6912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:37.969866 6912 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9eeeeba7f4140b4a832937b0fabd697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37333 } }
I20260807 18:42:37.970206 6912 leader_election.cc:302] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9eeeeba7f4140b4a832937b0fabd697; no voters:
I20260807 18:42:37.970561 6912 leader_election.cc:288] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:37.970983 6923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:37.971240 6923 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [term 1 LEADER]: Becoming Leader. State: Replica: a9eeeeba7f4140b4a832937b0fabd697, State: Running, Role: LEADER
I20260807 18:42:37.971683 6923 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9eeeeba7f4140b4a832937b0fabd697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37333 } }
I20260807 18:42:37.971979 6912 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:37.973456 6924 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9eeeeba7f4140b4a832937b0fabd697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9eeeeba7f4140b4a832937b0fabd697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37333 } } }
I20260807 18:42:37.973431 6925 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9eeeeba7f4140b4a832937b0fabd697. Latest consensus state: current_term: 1 leader_uuid: "a9eeeeba7f4140b4a832937b0fabd697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9eeeeba7f4140b4a832937b0fabd697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37333 } } }
I20260807 18:42:37.973588 6924 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:37.973588 6925 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:37.975488 6915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:37.970373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37185 (local address 127.0.0.1:37333)
0807 18:42:37.970512 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:37.970517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:37.970729 (+ 212us) server_negotiation.cc:407] Connection header received
0807 18:42:37.970870 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:37.970873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:37.970923 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:37.971004 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:37.971419 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:37.972170 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:37.973407 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:37.973774 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:37.973883 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:37.975009 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:37.975013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:37.975015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:37.975028 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:37.975064 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:37.975075 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:37.975159 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:37.975290 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 18:42:37.975345 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 18:42:37.977089 6942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:42:37.977155 6942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:42:37.977234 6940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:37.978109 6940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:37.983647 6940 catalog_manager.cc:1347] Generated new cluster ID: b96464a32985481fbfb1963396a41a25
I20260807 18:42:37.983713 6940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:37.992010 6940 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:37.993036 6940 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:38.001456 6940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9eeeeba7f4140b4a832937b0fabd697: Generated new TSK 0
I20260807 18:42:38.002323 6940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:38.017289 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 6791
2026-08-07T18:42:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1675 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:38 dist-test-slave-2d19 krb5kdc[6954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:38 dist-test-slave-2d19 krb5kdc[6954](info): set up 2 sockets
Aug 07 18:42:38 dist-test-slave-2d19 krb5kdc[6954](info): commencing operation
krb5kdc: starting...
W20260807 18:42:39.286579 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:39 dist-test-slave-2d19 krb5kdc[6954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:39.423969 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38211
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:39.543470 6974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:39.543781 6974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:39.543828 6974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:39.548081 6974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:39.548167 6974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:39.548188 6974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:39.548206 6974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:39.548251 6974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:39.554013 6974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:39.555179 6974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:39.556308 6974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:39.565332 6986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:39.565316 6988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:39.565346 6985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:39.565522 6974 server_base.cc:1034] running on GCE node
I20260807 18:42:39.566552 6974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:39.567713 6974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:39.568903 6974 hybrid_clock.cc:648] HybridClock initialized: now 1786128159568889 us; error 27 us; skew 500 ppm
Aug 07 18:42:39 dist-test-slave-2d19 krb5kdc[6954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128159, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:39.575444 6974 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:39.576514 6974 webserver.cc:460] Webserver started at http://127.0.0.1:45685/ using document root <none> and password file <none>
I20260807 18:42:39.577066 6974 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:39.577132 6974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:39.577319 6974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:39.579387 6974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "161950e3ab2844c4bd684aa4d114049b"
format_stamp: "Formatted at 2026-08-07 18:42:39 on dist-test-slave-2d19"
server_key: "e60dda512406399cdec861f48d843832"
server_key_iv: "b4bcbb4b92507f31620bff11e436e9ac"
server_key_version: "encryptionkey@0"
I20260807 18:42:39.579911 6974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "161950e3ab2844c4bd684aa4d114049b"
format_stamp: "Formatted at 2026-08-07 18:42:39 on dist-test-slave-2d19"
server_key: "e60dda512406399cdec861f48d843832"
server_key_iv: "b4bcbb4b92507f31620bff11e436e9ac"
server_key_version: "encryptionkey@0"
I20260807 18:42:39.583765 6974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:42:39.586552 6995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:39.587929 6974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:42:39.588059 6974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "161950e3ab2844c4bd684aa4d114049b"
format_stamp: "Formatted at 2026-08-07 18:42:39 on dist-test-slave-2d19"
server_key: "e60dda512406399cdec861f48d843832"
server_key_iv: "b4bcbb4b92507f31620bff11e436e9ac"
server_key_version: "encryptionkey@0"
I20260807 18:42:39.588157 6974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:39.593411 6974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:39.597012 6974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:39.597173 6974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:39.605201 6974 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38211
I20260807 18:42:39.605226 7075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38211 every 8 connection(s)
I20260807 18:42:39.606268 6974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:39.609163 7076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:39.612182 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 6974
I20260807 18:42:39.612407 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:39.612759 6697 external_mini_cluster.cc:1428] Setting key cc27f07b0e2c13b6f4e24bdea7ae1218
I20260807 18:42:39.615298 7076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b: Bootstrap starting.
I20260807 18:42:39.618111 7076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:39.618703 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:39.614334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47601 (local address 127.0.0.1:38211)
0807 18:42:39.614816 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:39.614835 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:39.614879 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:42:39.615704 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:39.615727 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:39.616082 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:39.616479 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:39.617288 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:39.617293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:39.617307 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:39.617320 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:39.617387 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:39.617416 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:39.617535 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:39.617630 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 18:42:39.617775 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":135,"threads_started":1}
I20260807 18:42:39.618983 7076 log.cc:824] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:39.621371 7076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b: No bootstrap required, opened a new log
I20260807 18:42:39.621877 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:39.625356 7076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161950e3ab2844c4bd684aa4d114049b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38211 } }
I20260807 18:42:39.625761 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:39.623996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33725 (local address 127.0.0.1:38211)
0807 18:42:39.624128 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:39.624133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:39.624404 (+ 271us) server_negotiation.cc:407] Connection header received
0807 18:42:39.624510 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:39.624513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:39.624546 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:39.624625 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:39.625349 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:39.625353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:39.625355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:39.625369 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:39.625399 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:39.625413 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:39.625514 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:39.625567 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 18:42:39.625606 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:42:39.625643 7076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:39.625937 7076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161950e3ab2844c4bd684aa4d114049b, State: Initialized, Role: FOLLOWER
I20260807 18:42:39.626371 7076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161950e3ab2844c4bd684aa4d114049b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38211 } }
I20260807 18:42:39.626492 7076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:39.626580 7076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:39.626700 7076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:39.627735 7076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161950e3ab2844c4bd684aa4d114049b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38211 } }
I20260807 18:42:39.628093 7076 leader_election.cc:302] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 161950e3ab2844c4bd684aa4d114049b; no voters:
I20260807 18:42:39.628425 7076 leader_election.cc:288] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:39.628571 7098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:39.628830 7098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [term 1 LEADER]: Becoming Leader. State: Replica: 161950e3ab2844c4bd684aa4d114049b, State: Running, Role: LEADER
I20260807 18:42:39.629179 7098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161950e3ab2844c4bd684aa4d114049b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38211 } }
I20260807 18:42:39.629719 7076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:39.631103 7102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 161950e3ab2844c4bd684aa4d114049b. Latest consensus state: current_term: 1 leader_uuid: "161950e3ab2844c4bd684aa4d114049b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161950e3ab2844c4bd684aa4d114049b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38211 } } }
I20260807 18:42:39.631244 7102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:39.631095 7100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "161950e3ab2844c4bd684aa4d114049b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "161950e3ab2844c4bd684aa4d114049b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38211 } } }
I20260807 18:42:39.631292 7100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:39.631636 7115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:39.634649 7115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:39.639779 7115 catalog_manager.cc:1347] Generated new cluster ID: 19a068e6df224712ab38f94c072383f9
I20260807 18:42:39.639844 7115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:39.650849 7115 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:39.651235 7115 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:39.658038 7115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 161950e3ab2844c4bd684aa4d114049b: Generated new TSK 0
I20260807 18:42:39.658859 7115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:39.671108 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 6974
2026-08-07T18:42:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1656 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:39 dist-test-slave-2d19 krb5kdc[7131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:39 dist-test-slave-2d19 krb5kdc[7131](info): set up 2 sockets
Aug 07 18:42:39 dist-test-slave-2d19 krb5kdc[7131](info): commencing operation
krb5kdc: starting...
W20260807 18:42:40.965848 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:41 dist-test-slave-2d19 krb5kdc[7131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:41.098994 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:41.214213 7149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:41.214529 7149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:41.214607 7149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:41.218572 7149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:41.218678 7149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:41.218716 7149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:41.218756 7149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:41.218791 7149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:41.224077 7149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:41.225212 7149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:41.226277 7149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:41.237282 7161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:41.237279 7156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:41.237782 7157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:41.237942 7149 server_base.cc:1034] running on GCE node
I20260807 18:42:41.238799 7149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:41.240434 7149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:41.241652 7149 hybrid_clock.cc:648] HybridClock initialized: now 1786128161241630 us; error 42 us; skew 500 ppm
Aug 07 18:42:41 dist-test-slave-2d19 krb5kdc[7131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128161, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:41.244596 7149 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:41.245550 7149 webserver.cc:460] Webserver started at http://127.0.0.1:46193/ using document root <none> and password file <none>
I20260807 18:42:41.246135 7149 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:41.246229 7149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:41.246471 7149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:41.248618 7149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "a5d646d1b282491fb7a498f9a97c8dad"
format_stamp: "Formatted at 2026-08-07 18:42:41 on dist-test-slave-2d19"
server_key: "d66a7a82cad36796ebe34899d2775a12"
server_key_iv: "a686ef631bb0eb0ad625ab1ed7c6439c"
server_key_version: "encryptionkey@0"
I20260807 18:42:41.249297 7149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "a5d646d1b282491fb7a498f9a97c8dad"
format_stamp: "Formatted at 2026-08-07 18:42:41 on dist-test-slave-2d19"
server_key: "d66a7a82cad36796ebe34899d2775a12"
server_key_iv: "a686ef631bb0eb0ad625ab1ed7c6439c"
server_key_version: "encryptionkey@0"
I20260807 18:42:41.253144 7149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:42:41.255615 7171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:41.256851 7149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:42:41.256992 7149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "a5d646d1b282491fb7a498f9a97c8dad"
format_stamp: "Formatted at 2026-08-07 18:42:41 on dist-test-slave-2d19"
server_key: "d66a7a82cad36796ebe34899d2775a12"
server_key_iv: "a686ef631bb0eb0ad625ab1ed7c6439c"
server_key_version: "encryptionkey@0"
I20260807 18:42:41.257107 7149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:41.268188 7149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:41.271531 7149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:41.271749 7149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:41.279058 7265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34811 every 8 connection(s)
I20260807 18:42:41.279059 7149 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34811
I20260807 18:42:41.280174 7149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:41.283175 7267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:41.286036 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7149
I20260807 18:42:41.286204 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:41.286607 6697 external_mini_cluster.cc:1428] Setting key fc4050a8e0f94dbcc1c962b3f85d7038
I20260807 18:42:41.289858 7267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad: Bootstrap starting.
I20260807 18:42:41.292761 7267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:41.293653 7267 log.cc:824] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:41.295773 7267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad: No bootstrap required, opened a new log
Aug 07 18:42:41 dist-test-slave-2d19 krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:42:41.300423 7267 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d646d1b282491fb7a498f9a97c8dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } }
I20260807 18:42:41.300731 7267 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:41.300802 7267 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5d646d1b282491fb7a498f9a97c8dad, State: Initialized, Role: FOLLOWER
I20260807 18:42:41.301340 7267 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d646d1b282491fb7a498f9a97c8dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } }
I20260807 18:42:41.301503 7267 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:41.301571 7267 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:41.301669 7267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:41.302925 7267 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d646d1b282491fb7a498f9a97c8dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } }
I20260807 18:42:41.303305 7267 leader_election.cc:302] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5d646d1b282491fb7a498f9a97c8dad; no voters:
I20260807 18:42:41.303630 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:41.288436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:51525 (local address 127.0.0.1:34811)
0807 18:42:41.288888 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:41.288909 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:41.288966 (+ 57us) server_negotiation.cc:407] Connection header received
0807 18:42:41.289756 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:41.289779 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:41.290235 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:41.290594 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:41.291294 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:41.292987 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:41.293942 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:41.294555 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:41.294720 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:41.297377 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:41.297384 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:41.297402 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:41.297437 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:41.299853 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:41.300498 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:41.300504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:41.300510 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:41.300652 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:41.301121 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:41.301124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:41.301126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:41.301675 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:41.301924 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:41.302271 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 18:42:41.302525 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":118,"threads_started":1}
I20260807 18:42:41.303683 7267 leader_election.cc:288] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:41.303812 7277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:41.304088 7277 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [term 1 LEADER]: Becoming Leader. State: Replica: a5d646d1b282491fb7a498f9a97c8dad, State: Running, Role: LEADER
I20260807 18:42:41.304576 7277 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d646d1b282491fb7a498f9a97c8dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } }
I20260807 18:42:41.305107 7267 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:41.305964 7277 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5d646d1b282491fb7a498f9a97c8dad. Latest consensus state: current_term: 1 leader_uuid: "a5d646d1b282491fb7a498f9a97c8dad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d646d1b282491fb7a498f9a97c8dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } } }
I20260807 18:42:41.306782 7277 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:41.306638 7279 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5d646d1b282491fb7a498f9a97c8dad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5d646d1b282491fb7a498f9a97c8dad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34811 } } }
I20260807 18:42:41.307113 7279 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:41.307405 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:41.307440 7298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:41.311214 7298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:41.317543 7298 catalog_manager.cc:1347] Generated new cluster ID: 32418f5193324ac492aa52e3728d0140
I20260807 18:42:41.317669 7298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:41.319212 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:41.309745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:50335 (local address 127.0.0.1:34811)
0807 18:42:41.309848 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:41.309852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:41.310178 (+ 326us) server_negotiation.cc:407] Connection header received
0807 18:42:41.310285 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:41.310288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:41.310339 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:41.310436 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:41.311306 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:41.312074 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:41.313289 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:41.313619 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:41.313733 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:41.315487 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:41.315490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:41.315493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:41.315519 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:41.317107 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:41.317722 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:41.317728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:41.317730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:41.317820 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:41.318265 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:41.318268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:41.318270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:41.318505 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:41.318666 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:41.318967 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 18:42:41.319068 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 18:42:41.331707 7298 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:41.332769 7298 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:41.337952 7298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5d646d1b282491fb7a498f9a97c8dad: Generated new TSK 0
I20260807 18:42:41.338615 7298 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:41.347983 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7149
W20260807 18:42:41.362687 7129 connection.cc:535] client connection to 127.0.0.1:34811 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34811): Connection reset by peer (error 104)
2026-08-07T18:42:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1672 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:41 dist-test-slave-2d19 krb5kdc[7323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:41 dist-test-slave-2d19 krb5kdc[7323](info): set up 2 sockets
Aug 07 18:42:41 dist-test-slave-2d19 krb5kdc[7323](info): commencing operation
krb5kdc: starting...
W20260807 18:42:42.693531 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:42 dist-test-slave-2d19 krb5kdc[7323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:42.827142 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:42.940968 7347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:42.941299 7347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:42.941393 7347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:42.946152 7347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:42.946269 7347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:42.946310 7347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:42.946352 7347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:42.946388 7347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:42.952054 7347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:42.953297 7347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:42.954380 7347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:42.963400 7367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:42.963387 7363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:42.963514 7362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:42.963779 7347 server_base.cc:1034] running on GCE node
I20260807 18:42:42.964474 7347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:42.965677 7347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:42.966863 7347 hybrid_clock.cc:648] HybridClock initialized: now 1786128162966828 us; error 46 us; skew 500 ppm
Aug 07 18:42:42 dist-test-slave-2d19 krb5kdc[7323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128162, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:42.969851 7347 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:42.970782 7347 webserver.cc:460] Webserver started at http://127.0.0.1:42769/ using document root <none> and password file <none>
I20260807 18:42:42.971366 7347 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:42.971460 7347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:42.971704 7347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:42.973907 7347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "767598ea3c11443a800c429c7ac3186a"
format_stamp: "Formatted at 2026-08-07 18:42:42 on dist-test-slave-2d19"
server_key: "57a6172e31b6d524cffb735327b8a44c"
server_key_iv: "a7c83921a04657368b27ff3ea40d1023"
server_key_version: "encryptionkey@0"
I20260807 18:42:42.974486 7347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "767598ea3c11443a800c429c7ac3186a"
format_stamp: "Formatted at 2026-08-07 18:42:42 on dist-test-slave-2d19"
server_key: "57a6172e31b6d524cffb735327b8a44c"
server_key_iv: "a7c83921a04657368b27ff3ea40d1023"
server_key_version: "encryptionkey@0"
I20260807 18:42:42.978461 7347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 18:42:42.981019 7376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:42.982260 7347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:42:42.982414 7347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "767598ea3c11443a800c429c7ac3186a"
format_stamp: "Formatted at 2026-08-07 18:42:42 on dist-test-slave-2d19"
server_key: "57a6172e31b6d524cffb735327b8a44c"
server_key_iv: "a7c83921a04657368b27ff3ea40d1023"
server_key_version: "encryptionkey@0"
I20260807 18:42:42.982533 7347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:43.004863 7347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:43.008502 7347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:43.008733 7347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:43.016039 7347 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41087
I20260807 18:42:43.016103 7458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41087 every 8 connection(s)
I20260807 18:42:43.017097 7347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:43.019948 7460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:43.024715 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7347
I20260807 18:42:43.024906 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:43.025214 6697 external_mini_cluster.cc:1428] Setting key 7d8c3d041b9cff0ee5d159790d928e66
I20260807 18:42:43.025712 7460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a: Bootstrap starting.
I20260807 18:42:43.028038 7460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:43.028877 7460 log.cc:824] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:43.030783 7460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a: No bootstrap required, opened a new log
I20260807 18:42:43.034366 7460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767598ea3c11443a800c429c7ac3186a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260807 18:42:43.034617 7460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:43.034745 7460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 767598ea3c11443a800c429c7ac3186a, State: Initialized, Role: FOLLOWER
I20260807 18:42:43.034787 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:43.026634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:40095 (local address 127.0.0.1:41087)
0807 18:42:43.027079 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:43.027102 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:43.027322 (+ 220us) server_negotiation.cc:407] Connection header received
0807 18:42:43.028114 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:43.028133 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:43.028613 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:43.028960 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:43.029370 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:43.030339 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:43.031309 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:43.031786 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:43.031978 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:43.032949 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:43.032954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:43.032966 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:43.033346 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:43.033398 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:43.033419 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:43.033573 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:43.033727 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 18:42:43.033885 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":136,"threads_started":1}
I20260807 18:42:43.035296 7460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767598ea3c11443a800c429c7ac3186a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260807 18:42:43.035465 7460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:43.035532 7460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:43.035617 7460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:43.036609 7460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767598ea3c11443a800c429c7ac3186a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260807 18:42:43.036994 7460 leader_election.cc:302] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 767598ea3c11443a800c429c7ac3186a; no voters:
I20260807 18:42:43.037376 7460 leader_election.cc:288] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:43.037497 7467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:43.037721 7467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [term 1 LEADER]: Becoming Leader. State: Replica: 767598ea3c11443a800c429c7ac3186a, State: Running, Role: LEADER
I20260807 18:42:43.038121 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:43.038102 7467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767598ea3c11443a800c429c7ac3186a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260807 18:42:43.038690 7460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:43.039309 7467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 767598ea3c11443a800c429c7ac3186a. Latest consensus state: current_term: 1 leader_uuid: "767598ea3c11443a800c429c7ac3186a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767598ea3c11443a800c429c7ac3186a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } } }
I20260807 18:42:43.040100 7467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:43.039974 7468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "767598ea3c11443a800c429c7ac3186a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767598ea3c11443a800c429c7ac3186a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } } }
I20260807 18:42:43.040596 7468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a [sys.catalog]: This master's current role is: LEADER
W20260807 18:42:43.044196 7502 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:42:43.044302 7502 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:42:43.044421 7491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:43.045437 7491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:43.045910 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:43.041601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:55337 (local address 127.0.0.1:41087)
0807 18:42:43.041707 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:43.041711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:43.041898 (+ 187us) server_negotiation.cc:407] Connection header received
0807 18:42:43.042060 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:43.042063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:43.042113 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:43.042196 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:43.042460 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:43.042973 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:43.043797 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:43.044053 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:43.044163 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:43.045262 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:43.045266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:43.045269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:43.045283 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:43.045311 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:43.045325 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:43.045431 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:43.045698 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 18:42:43.045741 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 18:42:43.051301 7491 catalog_manager.cc:1347] Generated new cluster ID: 45fcb2af1278479187148aeea35edc1f
I20260807 18:42:43.051388 7491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:43.065336 7491 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:43.066290 7491 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:43.071884 7491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 767598ea3c11443a800c429c7ac3186a: Generated new TSK 0
I20260807 18:42:43.072594 7491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:43.083096 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7347
W20260807 18:42:43.110491 7321 connection.cc:535] client connection to 127.0.0.1:41087 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41087): Connection reset by peer (error 104)
2026-08-07T18:42:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1747 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:43 dist-test-slave-2d19 krb5kdc[7513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:43 dist-test-slave-2d19 krb5kdc[7513](info): set up 2 sockets
Aug 07 18:42:43 dist-test-slave-2d19 krb5kdc[7513](info): commencing operation
krb5kdc: starting...
W20260807 18:42:44.354272 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:44 dist-test-slave-2d19 krb5kdc[7513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:44.488495 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:44.603418 7542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:44.603729 7542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:44.603801 7542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:44.607776 7542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:44.607882 7542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:44.607921 7542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:44.607957 7542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:44.607986 7542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:44.613268 7542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:44.614333 7542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:44.615370 7542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:44.624380 7557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:44.624367 7553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:44.624439 7542 server_base.cc:1034] running on GCE node
W20260807 18:42:44.624361 7554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:44.625368 7542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:44.626506 7542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:44.627673 7542 hybrid_clock.cc:648] HybridClock initialized: now 1786128164627655 us; error 32 us; skew 500 ppm
Aug 07 18:42:44 dist-test-slave-2d19 krb5kdc[7513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:44.635245 7542 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:44.636215 7542 webserver.cc:460] Webserver started at http://127.0.0.1:35585/ using document root <none> and password file <none>
I20260807 18:42:44.636831 7542 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:44.636895 7542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:44.637077 7542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:44.639271 7542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "2baf9fd313b14f3a95d3beb8217816bb"
format_stamp: "Formatted at 2026-08-07 18:42:44 on dist-test-slave-2d19"
server_key: "cff48862e4ffe14a45f9d0b0b997d166"
server_key_iv: "1e07c9096c6d97124aac074e41e89ff3"
server_key_version: "encryptionkey@0"
I20260807 18:42:44.639797 7542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "2baf9fd313b14f3a95d3beb8217816bb"
format_stamp: "Formatted at 2026-08-07 18:42:44 on dist-test-slave-2d19"
server_key: "cff48862e4ffe14a45f9d0b0b997d166"
server_key_iv: "1e07c9096c6d97124aac074e41e89ff3"
server_key_version: "encryptionkey@0"
I20260807 18:42:44.643711 7542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:42:44.646278 7576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:44.647572 7542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:42:44.647732 7542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "2baf9fd313b14f3a95d3beb8217816bb"
format_stamp: "Formatted at 2026-08-07 18:42:44 on dist-test-slave-2d19"
server_key: "cff48862e4ffe14a45f9d0b0b997d166"
server_key_iv: "1e07c9096c6d97124aac074e41e89ff3"
server_key_version: "encryptionkey@0"
I20260807 18:42:44.647851 7542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:44.652678 7542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:44.656013 7542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:44.656219 7542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:44.663406 7542 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36889
I20260807 18:42:44.663424 7659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36889 every 8 connection(s)
I20260807 18:42:44.664382 7542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:44.666265 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7542
I20260807 18:42:44.666425 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:44.666746 6697 external_mini_cluster.cc:1428] Setting key e5dea248ced5cb606fd3fa9a93bdfb4c
I20260807 18:42:44.667620 7661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:44.672034 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:44.668108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:52923 (local address 127.0.0.1:36889)
0807 18:42:44.668590 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:44.668606 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:44.668642 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:42:44.669278 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:44.669294 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:44.669580 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:44.669864 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:44.670752 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:44.670757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:44.670769 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:44.670780 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:44.670835 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:44.670854 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:44.670965 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:44.671103 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 18:42:44.671285 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":195,"threads_started":1}
I20260807 18:42:44.674305 7661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb: Bootstrap starting.
I20260807 18:42:44.675002 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:44.676653 7661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:44.677565 7661 log.cc:824] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:44.679088 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:44.677164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:40959 (local address 127.0.0.1:36889)
0807 18:42:44.677294 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:44.677299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:44.677438 (+ 139us) server_negotiation.cc:407] Connection header received
0807 18:42:44.677537 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:44.677540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:44.677580 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:44.677674 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:44.678641 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:44.678645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:44.678647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:44.678659 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:44.678686 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:44.678703 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:44.678796 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:44.678925 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 18:42:44.678965 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:42:44.679978 7661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb: No bootstrap required, opened a new log
I20260807 18:42:44.683538 7661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baf9fd313b14f3a95d3beb8217816bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260807 18:42:44.683797 7661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:44.683863 7661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2baf9fd313b14f3a95d3beb8217816bb, State: Initialized, Role: FOLLOWER
I20260807 18:42:44.684388 7661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baf9fd313b14f3a95d3beb8217816bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260807 18:42:44.684561 7661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:44.684656 7661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:44.684760 7661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:44.685684 7661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baf9fd313b14f3a95d3beb8217816bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260807 18:42:44.686026 7661 leader_election.cc:302] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2baf9fd313b14f3a95d3beb8217816bb; no voters:
I20260807 18:42:44.686348 7661 leader_election.cc:288] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:44.686529 7676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:44.686775 7676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [term 1 LEADER]: Becoming Leader. State: Replica: 2baf9fd313b14f3a95d3beb8217816bb, State: Running, Role: LEADER
I20260807 18:42:44.687129 7676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baf9fd313b14f3a95d3beb8217816bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } }
I20260807 18:42:44.687674 7661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:44.688882 7678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2baf9fd313b14f3a95d3beb8217816bb. Latest consensus state: current_term: 1 leader_uuid: "2baf9fd313b14f3a95d3beb8217816bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baf9fd313b14f3a95d3beb8217816bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } } }
I20260807 18:42:44.689023 7678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:44.689237 7677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2baf9fd313b14f3a95d3beb8217816bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2baf9fd313b14f3a95d3beb8217816bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36889 } } }
I20260807 18:42:44.689388 7677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:44.689471 7694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:44.692783 7694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:44.698516 7694 catalog_manager.cc:1347] Generated new cluster ID: 4a3f648447394c60b2d427542037c436
I20260807 18:42:44.698601 7694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:44.709697 7694 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:44.710000 7694 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:44.718272 7694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2baf9fd313b14f3a95d3beb8217816bb: Generated new TSK 0
I20260807 18:42:44.718947 7694 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:44.724372 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7542
2026-08-07T18:42:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1629 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:44 dist-test-slave-2d19 krb5kdc[7712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:44 dist-test-slave-2d19 krb5kdc[7712](info): set up 2 sockets
Aug 07 18:42:44 dist-test-slave-2d19 krb5kdc[7712](info): commencing operation
krb5kdc: starting...
W20260807 18:42:45.999362 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:46 dist-test-slave-2d19 krb5kdc[7712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:46.140663 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:46.257170 7734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:46.257450 7734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:46.257496 7734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:46.261468 7734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:46.261535 7734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:46.261555 7734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:46.261572 7734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:46.261588 7734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:46.266934 7734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:46.267975 7734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:46.268993 7734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:42:46.278488 7734 server_base.cc:1034] running on GCE node
W20260807 18:42:46.278473 7748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:46.278496 7744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:46.278452 7743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:46.279382 7734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:46.280769 7734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:46.282009 7734 hybrid_clock.cc:648] HybridClock initialized: now 1786128166281976 us; error 47 us; skew 500 ppm
Aug 07 18:42:46 dist-test-slave-2d19 krb5kdc[7712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128166, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:46.285210 7734 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:46.286317 7734 webserver.cc:460] Webserver started at http://127.0.0.1:36817/ using document root <none> and password file <none>
I20260807 18:42:46.286998 7734 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:46.287144 7734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:46.287426 7734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:46.289808 7734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "c6789e326dc34c1d910d3f7c735adc0c"
format_stamp: "Formatted at 2026-08-07 18:42:46 on dist-test-slave-2d19"
server_key: "002424fa5b6d23f3d71a4985b4a16f08"
server_key_iv: "55e8002fb09d2249ccdc768122e7ff23"
server_key_version: "encryptionkey@0"
I20260807 18:42:46.290452 7734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "c6789e326dc34c1d910d3f7c735adc0c"
format_stamp: "Formatted at 2026-08-07 18:42:46 on dist-test-slave-2d19"
server_key: "002424fa5b6d23f3d71a4985b4a16f08"
server_key_iv: "55e8002fb09d2249ccdc768122e7ff23"
server_key_version: "encryptionkey@0"
I20260807 18:42:46.294740 7734 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:42:46.297640 7760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:46.299147 7734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:42:46.299314 7734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "c6789e326dc34c1d910d3f7c735adc0c"
format_stamp: "Formatted at 2026-08-07 18:42:46 on dist-test-slave-2d19"
server_key: "002424fa5b6d23f3d71a4985b4a16f08"
server_key_iv: "55e8002fb09d2249ccdc768122e7ff23"
server_key_version: "encryptionkey@0"
I20260807 18:42:46.299455 7734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:46.313409 7734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:46.316664 7734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:46.316897 7734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:46.324774 7734 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40269
I20260807 18:42:46.324783 7850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40269 every 8 connection(s)
I20260807 18:42:46.325811 7734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:46.328789 7852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:46.329070 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7734
I20260807 18:42:46.329176 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:46.329407 6697 external_mini_cluster.cc:1428] Setting key 2a0e0ed0714709d9fd3063af9e8b4522
I20260807 18:42:46.336064 7852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c: Bootstrap starting.
I20260807 18:42:46.338691 7852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:46.339597 7852 log.cc:824] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:42:46 dist-test-slave-2d19 krb5kdc[7712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:42:46.342300 7852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c: No bootstrap required, opened a new log
I20260807 18:42:46.345876 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:46.330815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:46503 (local address 127.0.0.1:40269)
0807 18:42:46.331243 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:46.331264 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:46.331314 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:42:46.332138 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:46.332163 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:46.332657 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:46.333040 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:46.333792 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:46.335311 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:46.336438 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:46.337138 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:46.337320 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:46.340502 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:46.340509 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:46.340527 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:46.340567 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:46.342515 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:46.343205 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:46.343210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:46.343215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:46.343330 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:46.343757 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:46.343761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:46.343762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:46.344210 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:46.344509 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:46.344868 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 18:42:46.345114 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":115,"threads_started":1}
I20260807 18:42:46.347265 7852 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6789e326dc34c1d910d3f7c735adc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } }
I20260807 18:42:46.347577 7852 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:46.347633 7852 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6789e326dc34c1d910d3f7c735adc0c, State: Initialized, Role: FOLLOWER
I20260807 18:42:46.348165 7852 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6789e326dc34c1d910d3f7c735adc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } }
I20260807 18:42:46.348385 7852 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:46.348474 7852 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:46.348595 7852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:46.349370 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:46.349941 7852 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6789e326dc34c1d910d3f7c735adc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } }
I20260807 18:42:46.350347 7852 leader_election.cc:302] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6789e326dc34c1d910d3f7c735adc0c; no voters:
I20260807 18:42:46.350740 7852 leader_election.cc:288] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:46.350813 7863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:46.351083 7863 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [term 1 LEADER]: Becoming Leader. State: Replica: c6789e326dc34c1d910d3f7c735adc0c, State: Running, Role: LEADER
I20260807 18:42:46.351506 7863 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6789e326dc34c1d910d3f7c735adc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } }
I20260807 18:42:46.352352 7852 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:46.353673 7868 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6789e326dc34c1d910d3f7c735adc0c. Latest consensus state: current_term: 1 leader_uuid: "c6789e326dc34c1d910d3f7c735adc0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6789e326dc34c1d910d3f7c735adc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } } }
I20260807 18:42:46.353824 7868 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:46.354214 7867 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6789e326dc34c1d910d3f7c735adc0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6789e326dc34c1d910d3f7c735adc0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } } }
I20260807 18:42:46.355723 7867 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c [sys.catalog]: This master's current role is: LEADER
W20260807 18:42:46.358857 7908 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:42:46.359016 7908 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:42:46.359149 7890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:46.360148 7890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:46.361192 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:46.351808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:34745 (local address 127.0.0.1:40269)
0807 18:42:46.351955 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:46.351959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:46.352143 (+ 184us) server_negotiation.cc:407] Connection header received
0807 18:42:46.352305 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:46.352308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:46.352355 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:46.352475 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:46.353307 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:46.354056 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:46.355169 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:46.355543 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:46.355659 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:46.357351 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:46.357355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:46.357358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:46.357383 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:46.358947 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:46.359546 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:46.359550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:46.359552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:46.359653 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:46.360077 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:46.360081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:46.360083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:46.360372 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:46.360559 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:46.360885 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 18:42:46.361006 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:42:46.366384 7890 catalog_manager.cc:1347] Generated new cluster ID: c0d3f8fa01f3493e81d5d38fc0c1cde2
I20260807 18:42:46.366482 7890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:46.385262 7890 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:46.386271 7890 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:46.393352 7890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6789e326dc34c1d910d3f7c735adc0c: Generated new TSK 0
I20260807 18:42:46.394145 7890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:46.399649 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7734
W20260807 18:42:46.415360 7708 connection.cc:535] client connection to 127.0.0.1:40269 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40269): Connection reset by peer (error 104)
2026-08-07T18:42:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1675 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:46 dist-test-slave-2d19 krb5kdc[7919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:46 dist-test-slave-2d19 krb5kdc[7919](info): set up 2 sockets
Aug 07 18:42:46 dist-test-slave-2d19 krb5kdc[7919](info): commencing operation
krb5kdc: starting...
W20260807 18:42:47.677453 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:47 dist-test-slave-2d19 krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:47.807852 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:47.923969 7940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:47.924320 7940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:47.924439 7940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:47.928421 7940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:47.928512 7940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:47.928567 7940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:47.928611 7940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:47.928661 7940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:47.934026 7940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45875
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:47.935122 7940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:47.936174 7940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:47.944959 7947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:47.944947 7953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:47.945168 7940 server_base.cc:1034] running on GCE node
W20260807 18:42:47.944947 7950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:47.945886 7940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:47.947093 7940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:47.948307 7940 hybrid_clock.cc:648] HybridClock initialized: now 1786128167948272 us; error 44 us; skew 500 ppm
Aug 07 18:42:47 dist-test-slave-2d19 krb5kdc[7919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:47.951134 7940 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:47.952168 7940 webserver.cc:460] Webserver started at http://127.0.0.1:44185/ using document root <none> and password file <none>
I20260807 18:42:47.952787 7940 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:47.952888 7940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:47.953110 7940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:47.955227 7940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "82fb14fa64f74af6ae128318efcd5de5"
format_stamp: "Formatted at 2026-08-07 18:42:47 on dist-test-slave-2d19"
server_key: "0162f1db2b405d164d306b20814b97c3"
server_key_iv: "250d3258165f8fe7880e45e1da99702b"
server_key_version: "encryptionkey@0"
I20260807 18:42:47.955821 7940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "82fb14fa64f74af6ae128318efcd5de5"
format_stamp: "Formatted at 2026-08-07 18:42:47 on dist-test-slave-2d19"
server_key: "0162f1db2b405d164d306b20814b97c3"
server_key_iv: "250d3258165f8fe7880e45e1da99702b"
server_key_version: "encryptionkey@0"
I20260807 18:42:47.959666 7940 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:42:47.962282 7964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:47.963557 7940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:42:47.963716 7940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "82fb14fa64f74af6ae128318efcd5de5"
format_stamp: "Formatted at 2026-08-07 18:42:47 on dist-test-slave-2d19"
server_key: "0162f1db2b405d164d306b20814b97c3"
server_key_iv: "250d3258165f8fe7880e45e1da99702b"
server_key_version: "encryptionkey@0"
I20260807 18:42:47.963832 7940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:47.979943 7940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:47.983119 7940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:47.983336 7940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:47.990975 7940 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45875
I20260807 18:42:47.991020 8046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45875 every 8 connection(s)
I20260807 18:42:47.992028 7940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:47.994979 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 7940
I20260807 18:42:47.995013 8047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:47.995132 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:47.995469 6697 external_mini_cluster.cc:1428] Setting key 2b48dbf1016a773c671a410aab61bde9
I20260807 18:42:48.001612 8047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5: Bootstrap starting.
I20260807 18:42:48.004042 8047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:48.004875 8047 log.cc:824] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:48.005342 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:47.997000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:55217 (local address 127.0.0.1:45875)
0807 18:42:47.997510 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:47.997530 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:47.997567 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:42:47.998405 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:47.998425 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:47.998772 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:47.999212 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:47.999636 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:48.000705 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:48.001671 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:48.002212 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:48.002405 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:48.003431 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:48.003437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:48.003450 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:48.003844 (+ 394us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:48.003909 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:48.003932 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:48.004082 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:48.004361 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 18:42:48.004515 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":144,"threads_started":1}
I20260807 18:42:48.006904 8047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5: No bootstrap required, opened a new log
I20260807 18:42:48.008509 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:48.011539 8047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fb14fa64f74af6ae128318efcd5de5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } }
I20260807 18:42:48.011848 8047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:48.011950 8047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fb14fa64f74af6ae128318efcd5de5, State: Initialized, Role: FOLLOWER
I20260807 18:42:48.012435 8047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fb14fa64f74af6ae128318efcd5de5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } }
I20260807 18:42:48.012567 8047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:48.012652 8047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:48.012765 8047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:48.013983 8047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fb14fa64f74af6ae128318efcd5de5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } }
I20260807 18:42:48.014391 8047 leader_election.cc:302] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82fb14fa64f74af6ae128318efcd5de5; no voters:
I20260807 18:42:48.014788 8047 leader_election.cc:288] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:48.014905 8062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:48.015188 8062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [term 1 LEADER]: Becoming Leader. State: Replica: 82fb14fa64f74af6ae128318efcd5de5, State: Running, Role: LEADER
I20260807 18:42:48.015486 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:48.010922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:51451 (local address 127.0.0.1:45875)
0807 18:42:48.011065 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:48.011076 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:48.011200 (+ 124us) server_negotiation.cc:407] Connection header received
0807 18:42:48.011321 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:48.011323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:48.011362 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:48.011428 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:48.011741 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:48.012509 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:48.013677 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:48.013911 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:48.013999 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:48.014942 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:48.014946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:48.014949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:48.014964 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:48.014995 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:48.015010 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:48.015107 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:48.015319 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 18:42:48.015358 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:42:48.015663 8062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fb14fa64f74af6ae128318efcd5de5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } }
I20260807 18:42:48.016369 8047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:48.017228 8065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82fb14fa64f74af6ae128318efcd5de5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fb14fa64f74af6ae128318efcd5de5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } } }
I20260807 18:42:48.017441 8065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:48.017745 8064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82fb14fa64f74af6ae128318efcd5de5. Latest consensus state: current_term: 1 leader_uuid: "82fb14fa64f74af6ae128318efcd5de5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fb14fa64f74af6ae128318efcd5de5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } } }
I20260807 18:42:48.017863 8064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:48.017902 8076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:48.021255 8076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:48.027038 8076 catalog_manager.cc:1347] Generated new cluster ID: 6ab7c321ddd24f80b5ceeaaadcb2248a
I20260807 18:42:48.027128 8076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:48.039389 8076 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:48.040371 8076 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:48.045914 8076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82fb14fa64f74af6ae128318efcd5de5: Generated new TSK 0
I20260807 18:42:48.046725 8076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:48.055939 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 7940
W20260807 18:42:48.068892 7916 connection.cc:535] client connection to 127.0.0.1:45875 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45875): Connection reset by peer (error 104)
2026-08-07T18:42:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1653 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:48 dist-test-slave-2d19 krb5kdc[8096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:48 dist-test-slave-2d19 krb5kdc[8096](info): set up 2 sockets
Aug 07 18:42:48 dist-test-slave-2d19 krb5kdc[8096](info): commencing operation
krb5kdc: starting...
W20260807 18:42:49.391122 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:49 dist-test-slave-2d19 krb5kdc[8096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:42:49.532135 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42483
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:49.653560 8118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:49.653920 8118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:49.654006 8118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:49.658344 8118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:49.658461 8118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:49.658504 8118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:49.658562 8118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:49.658604 8118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:49.664443 8118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42483
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:49.665681 8118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:49.666797 8118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:49.676484 8131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:49.676492 8125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:49.676551 8118 server_base.cc:1034] running on GCE node
W20260807 18:42:49.676492 8126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:49.677372 8118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:49.678587 8118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:49.679775 8118 hybrid_clock.cc:648] HybridClock initialized: now 1786128169679730 us; error 56 us; skew 500 ppm
Aug 07 18:42:49 dist-test-slave-2d19 krb5kdc[8096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:49.686817 8118 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:42:49.687979 8118 webserver.cc:460] Webserver started at http://127.0.0.1:35223/ using document root <none> and password file <none>
I20260807 18:42:49.688704 8118 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:49.688807 8118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:49.689066 8118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:49.691305 8118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "84394dd19d4a4ce9a255c4d299ab0830"
format_stamp: "Formatted at 2026-08-07 18:42:49 on dist-test-slave-2d19"
server_key: "e6fbd7ca4f898834131c6cd86ac01617"
server_key_iv: "267d489efda3a43888bc422f0b701cfc"
server_key_version: "encryptionkey@0"
I20260807 18:42:49.691933 8118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "84394dd19d4a4ce9a255c4d299ab0830"
format_stamp: "Formatted at 2026-08-07 18:42:49 on dist-test-slave-2d19"
server_key: "e6fbd7ca4f898834131c6cd86ac01617"
server_key_iv: "267d489efda3a43888bc422f0b701cfc"
server_key_version: "encryptionkey@0"
I20260807 18:42:49.695919 8118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:42:49.698586 8143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:49.699869 8118 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:42:49.700023 8118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "84394dd19d4a4ce9a255c4d299ab0830"
format_stamp: "Formatted at 2026-08-07 18:42:49 on dist-test-slave-2d19"
server_key: "e6fbd7ca4f898834131c6cd86ac01617"
server_key_iv: "267d489efda3a43888bc422f0b701cfc"
server_key_version: "encryptionkey@0"
I20260807 18:42:49.700145 8118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:49.705045 8118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:49.708144 8118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:49.708483 8118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:49.716143 8118 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42483
I20260807 18:42:49.716173 8238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42483 every 8 connection(s)
I20260807 18:42:49.717226 8118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:49.720372 8241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:49.720794 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 8118
I20260807 18:42:49.720942 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:49.721275 6697 external_mini_cluster.cc:1428] Setting key ccd1fde065a3a21e393646f240ea3c3d
I20260807 18:42:49.727025 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:49.722843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:46397 (local address 127.0.0.1:42483)
0807 18:42:49.723354 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:49.723369 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:49.723405 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:42:49.724161 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:49.724182 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:49.724561 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:49.724917 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:49.725814 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:49.725819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:49.725833 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:49.725843 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:49.725901 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:49.725921 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:49.726083 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:49.726158 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 18:42:49.726327 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":180,"threads_started":1}
I20260807 18:42:49.728157 8241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830: Bootstrap starting.
I20260807 18:42:49.729988 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:49.730906 8241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:49.731781 8241 log.cc:824] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:49.733870 8241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830: No bootstrap required, opened a new log
I20260807 18:42:49.734237 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:49.732161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.12:40691 (local address 127.0.0.1:42483)
0807 18:42:49.732346 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:49.732351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:49.732607 (+ 256us) server_negotiation.cc:407] Connection header received
0807 18:42:49.732762 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:49.732765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:49.732798 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:49.732900 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:49.733769 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:49.733772 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:49.733775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:49.733788 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:49.733827 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:49.733838 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:49.733944 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:49.734040 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 18:42:49.734084 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:42:49.737869 8241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84394dd19d4a4ce9a255c4d299ab0830" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } }
I20260807 18:42:49.738134 8241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:49.738200 8241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84394dd19d4a4ce9a255c4d299ab0830, State: Initialized, Role: FOLLOWER
I20260807 18:42:49.738684 8241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84394dd19d4a4ce9a255c4d299ab0830" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } }
I20260807 18:42:49.738847 8241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:49.738917 8241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:49.739027 8241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:49.740311 8241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84394dd19d4a4ce9a255c4d299ab0830" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } }
I20260807 18:42:49.740662 8241 leader_election.cc:302] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84394dd19d4a4ce9a255c4d299ab0830; no voters:
I20260807 18:42:49.741034 8241 leader_election.cc:288] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:49.741173 8258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:49.741428 8258 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [term 1 LEADER]: Becoming Leader. State: Replica: 84394dd19d4a4ce9a255c4d299ab0830, State: Running, Role: LEADER
I20260807 18:42:49.741761 8258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84394dd19d4a4ce9a255c4d299ab0830" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } }
I20260807 18:42:49.742357 8241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:49.743646 8259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84394dd19d4a4ce9a255c4d299ab0830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84394dd19d4a4ce9a255c4d299ab0830" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } } }
I20260807 18:42:49.743661 8260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84394dd19d4a4ce9a255c4d299ab0830. Latest consensus state: current_term: 1 leader_uuid: "84394dd19d4a4ce9a255c4d299ab0830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84394dd19d4a4ce9a255c4d299ab0830" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42483 } } }
I20260807 18:42:49.743800 8260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:49.743801 8259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:49.744289 8275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:49.747056 8275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:49.753295 8275 catalog_manager.cc:1347] Generated new cluster ID: aa11098adeca41fd9e8a60a6666f9bb0
I20260807 18:42:49.753376 8275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:49.761291 8275 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:49.761627 8275 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:49.771693 8275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84394dd19d4a4ce9a255c4d299ab0830: Generated new TSK 0
I20260807 18:42:49.772571 8275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:49.778074 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 8118
2026-08-07T18:42:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1726 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:49 dist-test-slave-2d19 krb5kdc[8292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:49 dist-test-slave-2d19 krb5kdc[8292](info): set up 2 sockets
Aug 07 18:42:49 dist-test-slave-2d19 krb5kdc[8292](info): commencing operation
krb5kdc: starting...
W20260807 18:42:51.102229 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:51 dist-test-slave-2d19 krb5kdc[8292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:42:51.240326 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:45141
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:45141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:51.356029 8312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:51.356333 8312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:51.356413 8312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:51.360437 8312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:51.360509 8312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:51.360529 8312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:51.360545 8312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:51.360561 8312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:51.365926 8312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:45141
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:45141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:51.367131 8312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:51.368211 8312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:51.378261 8325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:51.378250 8321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:51.378250 8323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:51.378489 8312 server_base.cc:1034] running on GCE node
I20260807 18:42:51.379289 8312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:51.380620 8312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:51.381816 8312 hybrid_clock.cc:648] HybridClock initialized: now 1786128171381786 us; error 41 us; skew 500 ppm
Aug 07 18:42:51 dist-test-slave-2d19 krb5kdc[8292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128171, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:51.384989 8312 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:51.385838 8312 webserver.cc:460] Webserver started at http://127.6.138.126:45177/ using document root <none> and password file <none>
I20260807 18:42:51.386371 8312 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:51.386433 8312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:51.386600 8312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:51.388754 8312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "3404271b34e7490b9dc6933af56f5dc5"
format_stamp: "Formatted at 2026-08-07 18:42:51 on dist-test-slave-2d19"
server_key: "b8ab04bf4bccb894a7619a97ab31c36d"
server_key_iv: "071e0937ccab353ef12226e4a2aa746a"
server_key_version: "encryptionkey@0"
I20260807 18:42:51.389348 8312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "3404271b34e7490b9dc6933af56f5dc5"
format_stamp: "Formatted at 2026-08-07 18:42:51 on dist-test-slave-2d19"
server_key: "b8ab04bf4bccb894a7619a97ab31c36d"
server_key_iv: "071e0937ccab353ef12226e4a2aa746a"
server_key_version: "encryptionkey@0"
I20260807 18:42:51.393360 8312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:42:51.395974 8340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:51.397416 8312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:42:51.397557 8312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "3404271b34e7490b9dc6933af56f5dc5"
format_stamp: "Formatted at 2026-08-07 18:42:51 on dist-test-slave-2d19"
server_key: "b8ab04bf4bccb894a7619a97ab31c36d"
server_key_iv: "071e0937ccab353ef12226e4a2aa746a"
server_key_version: "encryptionkey@0"
I20260807 18:42:51.397680 8312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:51.407670 8312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:51.410890 8312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:51.411108 8312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:51.418496 8312 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:45141
I20260807 18:42:51.418587 8420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:45141 every 8 connection(s)
I20260807 18:42:51.419507 8312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:51.422453 8425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:51.427591 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 8312
I20260807 18:42:51.427754 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:51.428041 6697 external_mini_cluster.cc:1428] Setting key 92812e9561e692be8d4bb0bd811be947
I20260807 18:42:51.428416 8425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5: Bootstrap starting.
I20260807 18:42:51.431057 8425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:51.432041 8425 log.cc:824] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:51.434327 8425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5: No bootstrap required, opened a new log
Aug 07 18:42:51 dist-test-slave-2d19 krb5kdc[8292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:51.438817 8425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3404271b34e7490b9dc6933af56f5dc5" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45141 } }
I20260807 18:42:51.439092 8425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:51.439175 8425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3404271b34e7490b9dc6933af56f5dc5, State: Initialized, Role: FOLLOWER
I20260807 18:42:51.439679 8425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3404271b34e7490b9dc6933af56f5dc5" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45141 } }
I20260807 18:42:51.439837 8425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:51.439929 8425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:51.440060 8425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:51.441068 8425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3404271b34e7490b9dc6933af56f5dc5" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45141 } }
I20260807 18:42:51.441429 8425 leader_election.cc:302] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3404271b34e7490b9dc6933af56f5dc5; no voters:
I20260807 18:42:51.441789 8425 leader_election.cc:288] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:51.441864 8437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:51.442083 8437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [term 1 LEADER]: Becoming Leader. State: Replica: 3404271b34e7490b9dc6933af56f5dc5, State: Running, Role: LEADER
I20260807 18:42:51.442397 8437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3404271b34e7490b9dc6933af56f5dc5" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45141 } }
I20260807 18:42:51.443169 8425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:51.444072 8438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3404271b34e7490b9dc6933af56f5dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3404271b34e7490b9dc6933af56f5dc5" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45141 } } }
I20260807 18:42:51.444222 8438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:51.444308 8432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:51.429526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42335 (local address 127.6.138.126:45141)
0807 18:42:51.430021 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:51.430039 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:51.430078 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:42:51.430749 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:51.430766 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:51.431156 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:51.431471 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:51.432474 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:51.433648 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:51.434844 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:51.435339 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:51.435507 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:51.438450 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:51.438455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:51.438470 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:51.438499 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:51.440672 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:51.441224 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:51.441230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:51.441237 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:51.441366 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:51.441753 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:51.441757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:51.441759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:51.442355 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:51.442594 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:51.442980 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 18:42:51.443240 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":144,"threads_started":1}
I20260807 18:42:51.444504 8438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3404271b34e7490b9dc6933af56f5dc5. Latest consensus state: current_term: 1 leader_uuid: "3404271b34e7490b9dc6933af56f5dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3404271b34e7490b9dc6933af56f5dc5" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 45141 } } }
I20260807 18:42:51.444608 8438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:51.444900 8454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:51.447674 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:51.448402 8454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:51.454980 8454 catalog_manager.cc:1347] Generated new cluster ID: 2113f4eef5194cb9b9c09cf921170c69
I20260807 18:42:51.455062 8454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:51.460521 8432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:51.451072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53399 (local address 127.6.138.126:45141)
0807 18:42:51.451178 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:51.451182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:51.451432 (+ 250us) server_negotiation.cc:407] Connection header received
0807 18:42:51.451592 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:51.451595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:51.451643 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:51.451723 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:51.452653 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:51.453287 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:51.454199 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:51.454492 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:51.454631 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:51.456654 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:51.456658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:51.456660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:51.456685 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:51.458295 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:51.458841 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:51.458844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:51.458846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:51.458920 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:51.459256 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:51.459259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:51.459261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:51.459456 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:51.459578 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:51.460236 (+ 658us) server_negotiation.cc:299] Negotiation successful
0807 18:42:51.460353 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:42:51.479414 8454 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:51.480759 8454 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:51.487854 8454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3404271b34e7490b9dc6933af56f5dc5: Generated new TSK 0
I20260807 18:42:51.488729 8454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:51.494686 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 8312
W20260807 18:42:51.507123 8290 connection.cc:535] client connection to 127.6.138.126:45141 recv error: Network error: failed to read from TLS socket (remote: 127.6.138.126:45141): Connection reset by peer (error 104)
2026-08-07T18:42:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1711 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:51 dist-test-slave-2d19 krb5kdc[8475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:51 dist-test-slave-2d19 krb5kdc[8475](info): set up 2 sockets
Aug 07 18:42:51 dist-test-slave-2d19 krb5kdc[8475](info): commencing operation
krb5kdc: starting...
W20260807 18:42:52.782125 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:52 dist-test-slave-2d19 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:42:52.928176 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37363
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:37363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:53.058182 8496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:53.058542 8496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:53.058629 8496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:53.063328 8496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:53.063436 8496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:53.063480 8496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:53.063545 8496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:53.063591 8496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:53.069681 8496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:37363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:53.070886 8496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:53.072057 8496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:53.081477 8504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:53.081512 8505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:53.081746 8508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:53.082010 8496 server_base.cc:1034] running on GCE node
I20260807 18:42:53.082655 8496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:53.083745 8496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:53.084944 8496 hybrid_clock.cc:648] HybridClock initialized: now 1786128173084920 us; error 40 us; skew 500 ppm
Aug 07 18:42:53 dist-test-slave-2d19 krb5kdc[8475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128173, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:53.087857 8496 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:53.088907 8496 webserver.cc:460] Webserver started at http://127.6.138.126:45615/ using document root <none> and password file <none>
I20260807 18:42:53.089501 8496 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:53.089563 8496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:53.089815 8496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:53.091861 8496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "fbe40e2087c243b59112aa96996aa79f"
format_stamp: "Formatted at 2026-08-07 18:42:53 on dist-test-slave-2d19"
server_key: "3bf4aea9dbdc0ccd8248dcac792cfb72"
server_key_iv: "4e8be2717adcbf4b6b341b3eda88a597"
server_key_version: "encryptionkey@0"
I20260807 18:42:53.092497 8496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "fbe40e2087c243b59112aa96996aa79f"
format_stamp: "Formatted at 2026-08-07 18:42:53 on dist-test-slave-2d19"
server_key: "3bf4aea9dbdc0ccd8248dcac792cfb72"
server_key_iv: "4e8be2717adcbf4b6b341b3eda88a597"
server_key_version: "encryptionkey@0"
I20260807 18:42:53.096524 8496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:42:53.099159 8520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:53.100661 8496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:42:53.100803 8496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "fbe40e2087c243b59112aa96996aa79f"
format_stamp: "Formatted at 2026-08-07 18:42:53 on dist-test-slave-2d19"
server_key: "3bf4aea9dbdc0ccd8248dcac792cfb72"
server_key_iv: "4e8be2717adcbf4b6b341b3eda88a597"
server_key_version: "encryptionkey@0"
I20260807 18:42:53.100922 8496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:53.130857 8496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:53.134841 8496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:53.135103 8496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:53.143685 8496 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:37363
I20260807 18:42:53.143694 8612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:37363 every 8 connection(s)
I20260807 18:42:53.144786 8496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:53.146131 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 8496
I20260807 18:42:53.146319 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:53.146637 6697 external_mini_cluster.cc:1428] Setting key 11de8483f1f626e7a862f6865306d158
I20260807 18:42:53.148156 8614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:53.154824 8614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f: Bootstrap starting.
I20260807 18:42:53.155922 8617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:53.148022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45693 (local address 127.6.138.126:37363)
0807 18:42:53.148521 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:53.148537 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:53.148721 (+ 184us) server_negotiation.cc:407] Connection header received
0807 18:42:53.149464 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:53.149481 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:53.149813 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:53.150110 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:53.150452 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:53.151417 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:53.152651 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:53.153125 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:53.153241 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:53.154224 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:53.154230 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:53.154242 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:53.154624 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:53.154678 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:53.154700 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:53.154800 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:53.155066 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 18:42:53.155210 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":196,"threads_started":1}
I20260807 18:42:53.157183 8614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:53.158073 8614 log.cc:824] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:53.158947 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:53.160198 8614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f: No bootstrap required, opened a new log
I20260807 18:42:53.165272 8617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:53.161054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46653 (local address 127.6.138.126:37363)
0807 18:42:53.161167 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:53.161171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:53.161350 (+ 179us) server_negotiation.cc:407] Connection header received
0807 18:42:53.161535 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:53.161537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:53.161582 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:53.161647 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:53.161958 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:53.162443 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:53.163666 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:53.163872 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:53.163949 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:53.164798 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:53.164801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:53.164803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:53.164813 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:53.164837 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:53.164847 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:53.164915 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:53.165083 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 18:42:53.165109 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:42:53.165429 8614 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe40e2087c243b59112aa96996aa79f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37363 } }
I20260807 18:42:53.165746 8614 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:53.165813 8614 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbe40e2087c243b59112aa96996aa79f, State: Initialized, Role: FOLLOWER
I20260807 18:42:53.166312 8614 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe40e2087c243b59112aa96996aa79f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37363 } }
I20260807 18:42:53.166499 8614 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:53.166577 8614 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:53.166682 8614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:53.167588 8614 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe40e2087c243b59112aa96996aa79f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37363 } }
I20260807 18:42:53.167939 8614 leader_election.cc:302] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbe40e2087c243b59112aa96996aa79f; no voters:
I20260807 18:42:53.168293 8614 leader_election.cc:288] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:53.168427 8630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:53.168727 8630 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [term 1 LEADER]: Becoming Leader. State: Replica: fbe40e2087c243b59112aa96996aa79f, State: Running, Role: LEADER
I20260807 18:42:53.169021 8630 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe40e2087c243b59112aa96996aa79f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37363 } }
I20260807 18:42:53.169587 8614 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:53.170758 8635 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbe40e2087c243b59112aa96996aa79f. Latest consensus state: current_term: 1 leader_uuid: "fbe40e2087c243b59112aa96996aa79f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe40e2087c243b59112aa96996aa79f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37363 } } }
I20260807 18:42:53.170895 8635 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:53.171187 8631 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbe40e2087c243b59112aa96996aa79f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbe40e2087c243b59112aa96996aa79f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 37363 } } }
I20260807 18:42:53.171286 8631 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:53.171295 8644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:53.174499 8644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:53.180399 8644 catalog_manager.cc:1347] Generated new cluster ID: 509b2877f8f64fff8306ed14a07dd277
I20260807 18:42:53.180523 8644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:53.193948 8644 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:53.194958 8644 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:53.201257 8644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbe40e2087c243b59112aa96996aa79f: Generated new TSK 0
I20260807 18:42:53.201997 8644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:53.212392 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 8496
W20260807 18:42:53.223356 8472 connection.cc:535] client connection to 127.6.138.126:37363 recv error: Network error: failed to read from TLS socket (remote: 127.6.138.126:37363): Connection reset by peer (error 104)
2026-08-07T18:42:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1717 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:53 dist-test-slave-2d19 krb5kdc[8660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:53 dist-test-slave-2d19 krb5kdc[8660](info): set up 2 sockets
Aug 07 18:42:53 dist-test-slave-2d19 krb5kdc[8660](info): commencing operation
krb5kdc: starting...
W20260807 18:42:54.478699 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:54 dist-test-slave-2d19 krb5kdc[8660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:42:54.616185 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42741
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:54.731225 8685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:54.731535 8685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:54.731621 8685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:54.735658 8685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:54.735742 8685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:54.735791 8685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:54.735847 8685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:54.735885 8685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:54.741273 8685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:54.742364 8685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:54.743410 8685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:54.752429 8694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:54.752547 8685 server_base.cc:1034] running on GCE node
W20260807 18:42:54.752437 8698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:54.752560 8695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:54.753294 8685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:54.754338 8685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:54.755508 8685 hybrid_clock.cc:648] HybridClock initialized: now 1786128174755494 us; error 27 us; skew 500 ppm
Aug 07 18:42:54 dist-test-slave-2d19 krb5kdc[8660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128174, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:54.763451 8685 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:54.764506 8685 webserver.cc:460] Webserver started at http://127.6.138.126:33657/ using document root <none> and password file <none>
I20260807 18:42:54.765101 8685 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:54.765164 8685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:54.765383 8685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:54.767419 8685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "c4cbb0b50c6542088bb858f0b130d368"
format_stamp: "Formatted at 2026-08-07 18:42:54 on dist-test-slave-2d19"
server_key: "1743174eb54cf11ddb39363711fe85bf"
server_key_iv: "86d6a597da3868baa030b7e749d6cb2d"
server_key_version: "encryptionkey@0"
I20260807 18:42:54.768003 8685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "c4cbb0b50c6542088bb858f0b130d368"
format_stamp: "Formatted at 2026-08-07 18:42:54 on dist-test-slave-2d19"
server_key: "1743174eb54cf11ddb39363711fe85bf"
server_key_iv: "86d6a597da3868baa030b7e749d6cb2d"
server_key_version: "encryptionkey@0"
I20260807 18:42:54.771777 8685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:42:54.774351 8711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:54.775652 8685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:42:54.775810 8685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "c4cbb0b50c6542088bb858f0b130d368"
format_stamp: "Formatted at 2026-08-07 18:42:54 on dist-test-slave-2d19"
server_key: "1743174eb54cf11ddb39363711fe85bf"
server_key_iv: "86d6a597da3868baa030b7e749d6cb2d"
server_key_version: "encryptionkey@0"
I20260807 18:42:54.775923 8685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:54.780674 8685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:54.783802 8685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:54.784026 8685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:54.791473 8685 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:42741
I20260807 18:42:54.791507 8795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:42741 every 8 connection(s)
I20260807 18:42:54.792482 8685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:54.793268 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 8685
I20260807 18:42:54.793422 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:54.793738 6697 external_mini_cluster.cc:1428] Setting key 3d693d649f66db37f1131c1d3bd4af95
I20260807 18:42:54.795828 8796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:54.799218 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:54.795167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56375 (local address 127.6.138.126:42741)
0807 18:42:54.795652 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:54.795670 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:54.795710 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:42:54.796520 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:54.796540 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:54.796842 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:54.797206 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:54.798044 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:54.798049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:54.798061 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:54.798071 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:54.798126 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:54.798145 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:54.798297 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:54.798368 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 18:42:54.798536 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":154,"threads_started":1}
I20260807 18:42:54.802266 8796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368: Bootstrap starting.
I20260807 18:42:54.802357 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:54.805094 8796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:54.806042 8796 log.cc:824] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:54.806576 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:54.804782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56319 (local address 127.6.138.126:42741)
0807 18:42:54.804916 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:54.804922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:54.805097 (+ 175us) server_negotiation.cc:407] Connection header received
0807 18:42:54.805174 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:54.805177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:54.805214 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:54.805307 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:54.806056 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:54.806062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:54.806066 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:54.806087 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:54.806117 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:54.806134 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:54.806225 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:54.806401 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 18:42:54.806446 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:42:54.808341 8796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368: No bootstrap required, opened a new log
I20260807 18:42:54.811832 8796 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cbb0b50c6542088bb858f0b130d368" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42741 } }
I20260807 18:42:54.812172 8796 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:54.812310 8796 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4cbb0b50c6542088bb858f0b130d368, State: Initialized, Role: FOLLOWER
I20260807 18:42:54.812810 8796 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cbb0b50c6542088bb858f0b130d368" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42741 } }
I20260807 18:42:54.812968 8796 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:54.813036 8796 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:54.813133 8796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:54.814076 8796 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cbb0b50c6542088bb858f0b130d368" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42741 } }
I20260807 18:42:54.814422 8796 leader_election.cc:302] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4cbb0b50c6542088bb858f0b130d368; no voters:
I20260807 18:42:54.814759 8796 leader_election.cc:288] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:54.814851 8813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:54.815104 8813 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [term 1 LEADER]: Becoming Leader. State: Replica: c4cbb0b50c6542088bb858f0b130d368, State: Running, Role: LEADER
I20260807 18:42:54.815438 8813 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cbb0b50c6542088bb858f0b130d368" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42741 } }
I20260807 18:42:54.816035 8796 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:54.817171 8816 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4cbb0b50c6542088bb858f0b130d368. Latest consensus state: current_term: 1 leader_uuid: "c4cbb0b50c6542088bb858f0b130d368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cbb0b50c6542088bb858f0b130d368" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42741 } } }
I20260807 18:42:54.817298 8816 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:54.817646 8815 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4cbb0b50c6542088bb858f0b130d368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4cbb0b50c6542088bb858f0b130d368" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42741 } } }
I20260807 18:42:54.817740 8831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:54.818627 8815 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:54.821061 8831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:54.826903 8831 catalog_manager.cc:1347] Generated new cluster ID: 146f235a13c54b8abeade0f5a8f2bc92
I20260807 18:42:54.827036 8831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:54.835446 8831 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:54.835764 8831 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:54.844636 8831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4cbb0b50c6542088bb858f0b130d368: Generated new TSK 0
I20260807 18:42:54.845312 8831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:54.853826 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 8685
2026-08-07T18:42:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1644 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:54 dist-test-slave-2d19 krb5kdc[8856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:54 dist-test-slave-2d19 krb5kdc[8856](info): set up 2 sockets
Aug 07 18:42:54 dist-test-slave-2d19 krb5kdc[8856](info): commencing operation
krb5kdc: starting...
W20260807 18:42:56.173960 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:56 dist-test-slave-2d19 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:42:56.311646 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38281
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:38281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:56.428128 8875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:56.428435 8875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:56.428480 8875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:56.433133 8875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:56.433216 8875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:56.433238 8875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:56.433257 8875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:56.433274 8875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:56.438804 8875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:38281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:38281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:56.439888 8875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:56.440925 8875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:56.449707 8881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:56.449734 8882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:56.449707 8887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:56.449999 8875 server_base.cc:1034] running on GCE node
I20260807 18:42:56.450654 8875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:56.451756 8875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:56.452950 8875 hybrid_clock.cc:648] HybridClock initialized: now 1786128176452939 us; error 32 us; skew 500 ppm
Aug 07 18:42:56 dist-test-slave-2d19 krb5kdc[8856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128176, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:56.455770 8875 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:56.456710 8875 webserver.cc:460] Webserver started at http://127.6.138.126:46855/ using document root <none> and password file <none>
I20260807 18:42:56.457242 8875 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:56.457301 8875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:56.457472 8875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:56.459467 8875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "bb5637e45b524404b4901e1e9bd60d4f"
format_stamp: "Formatted at 2026-08-07 18:42:56 on dist-test-slave-2d19"
server_key: "dd28ab5f2a626710c387c98ca1c26025"
server_key_iv: "fff64c4b63836630b1b557a1878c5d19"
server_key_version: "encryptionkey@0"
I20260807 18:42:56.460000 8875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "bb5637e45b524404b4901e1e9bd60d4f"
format_stamp: "Formatted at 2026-08-07 18:42:56 on dist-test-slave-2d19"
server_key: "dd28ab5f2a626710c387c98ca1c26025"
server_key_iv: "fff64c4b63836630b1b557a1878c5d19"
server_key_version: "encryptionkey@0"
I20260807 18:42:56.463704 8875 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 18:42:56.466229 8898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:56.467589 8875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:42:56.467727 8875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "bb5637e45b524404b4901e1e9bd60d4f"
format_stamp: "Formatted at 2026-08-07 18:42:56 on dist-test-slave-2d19"
server_key: "dd28ab5f2a626710c387c98ca1c26025"
server_key_iv: "fff64c4b63836630b1b557a1878c5d19"
server_key_version: "encryptionkey@0"
I20260807 18:42:56.467832 8875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:56.484097 8875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:56.487613 8875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:56.487830 8875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:56.495707 8875 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:38281
I20260807 18:42:56.495723 8985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:38281 every 8 connection(s)
I20260807 18:42:56.496783 8875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:56.499583 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 8875
I20260807 18:42:56.499732 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:56.499953 8987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:56.500069 6697 external_mini_cluster.cc:1428] Setting key f702817500484d3ae9ade3a68be84a0f
I20260807 18:42:56.507405 8987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f: Bootstrap starting.
Aug 07 18:42:56 dist-test-slave-2d19 krb5kdc[8856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:42:56.510285 8987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:56.511217 8987 log.cc:824] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:56.513239 8987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f: No bootstrap required, opened a new log
I20260807 18:42:56.514813 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:56.501483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34151 (local address 127.6.138.126:38281)
0807 18:42:56.501913 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:56.501935 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:56.501971 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:42:56.502771 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:56.502789 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:56.503149 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:56.503430 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:56.504402 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:56.505386 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:56.506618 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:56.507081 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:56.507199 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:56.510134 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:56.510140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:56.510163 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:56.510195 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:56.511970 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:56.512493 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:56.512499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:56.512503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:56.512623 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:56.512921 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:56.512929 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:56.512930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:56.513337 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:56.513554 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:56.513807 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 18:42:56.514101 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":148,"threads_started":1}
I20260807 18:42:56.516980 8987 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5637e45b524404b4901e1e9bd60d4f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38281 } }
I20260807 18:42:56.517231 8987 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:56.517308 8987 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb5637e45b524404b4901e1e9bd60d4f, State: Initialized, Role: FOLLOWER
I20260807 18:42:56.517735 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:56.517772 8987 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5637e45b524404b4901e1e9bd60d4f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38281 } }
I20260807 18:42:56.517972 8987 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:56.518061 8987 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:56.518162 8987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:56.519124 8987 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5637e45b524404b4901e1e9bd60d4f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38281 } }
I20260807 18:42:56.519485 8987 leader_election.cc:302] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb5637e45b524404b4901e1e9bd60d4f; no voters:
I20260807 18:42:56.519809 8987 leader_election.cc:288] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:56.519924 8999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:56.520243 8999 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [term 1 LEADER]: Becoming Leader. State: Replica: bb5637e45b524404b4901e1e9bd60d4f, State: Running, Role: LEADER
I20260807 18:42:56.520622 8999 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5637e45b524404b4901e1e9bd60d4f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38281 } }
I20260807 18:42:56.521160 8987 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:56.522426 9003 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb5637e45b524404b4901e1e9bd60d4f. Latest consensus state: current_term: 1 leader_uuid: "bb5637e45b524404b4901e1e9bd60d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5637e45b524404b4901e1e9bd60d4f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38281 } } }
I20260807 18:42:56.522426 9002 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb5637e45b524404b4901e1e9bd60d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5637e45b524404b4901e1e9bd60d4f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 38281 } } }
I20260807 18:42:56.522611 9003 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:56.522634 9002 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f [sys.catalog]: This master's current role is: LEADER
W20260807 18:42:56.526269 9024 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:42:56.526360 9024 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:42:56.526470 9013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:56.527326 9013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:56.532435 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:56.519849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44549 (local address 127.6.138.126:38281)
0807 18:42:56.519990 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:56.519995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:56.523617 (+ 3622us) server_negotiation.cc:407] Connection header received
0807 18:42:56.523982 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:56.523986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:56.524045 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:56.524086 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:56.524896 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:56.525668 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:56.526821 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:56.527150 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:56.527259 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:56.529070 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:56.529081 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:56.529084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:42:56.529112 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:56.530588 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:56.531035 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:56.531039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:56.531040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:56.531147 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:42:56.531489 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:42:56.531496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:42:56.531497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:42:56.531725 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:56.531881 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:56.532129 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 18:42:56.532223 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:42:56.533896 9013 catalog_manager.cc:1347] Generated new cluster ID: 4324dcda0cc542d5a9b41aaa3e2c016b
I20260807 18:42:56.533967 9013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:56.542447 9013 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:56.543509 9013 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:56.549513 9013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb5637e45b524404b4901e1e9bd60d4f: Generated new TSK 0
I20260807 18:42:56.550203 9013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:56.554630 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 8875
2026-08-07T18:42:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1700 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:56 dist-test-slave-2d19 krb5kdc[9032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:56 dist-test-slave-2d19 krb5kdc[9032](info): set up 2 sockets
Aug 07 18:42:56 dist-test-slave-2d19 krb5kdc[9032](info): commencing operation
krb5kdc: starting...
W20260807 18:42:57.817870 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:57 dist-test-slave-2d19 krb5kdc[9032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:42:57.960421 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:40471
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:34687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:40471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:58.081090 9058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:58.081435 9058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:58.081516 9058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:58.085640 9058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:58.085733 9058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:58.085773 9058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:58.085834 9058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:58.085873 9058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:58.091349 9058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:34687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:40471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:40471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:58.092531 9058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:58.093585 9058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:58.102669 9065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:58.102739 9058 server_base.cc:1034] running on GCE node
W20260807 18:42:58.102669 9064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:58.102677 9067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:58.103587 9058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:58.104833 9058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:58.106024 9058 hybrid_clock.cc:648] HybridClock initialized: now 1786128178105977 us; error 58 us; skew 500 ppm
Aug 07 18:42:58 dist-test-slave-2d19 krb5kdc[9032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128178, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:58.109130 9058 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:58.110095 9058 webserver.cc:460] Webserver started at http://127.6.138.126:36453/ using document root <none> and password file <none>
I20260807 18:42:58.110767 9058 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:58.110865 9058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:58.111123 9058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:58.113461 9058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "47b7db9766544b76a77f946b79233a1f"
format_stamp: "Formatted at 2026-08-07 18:42:58 on dist-test-slave-2d19"
server_key: "c3fbcbe2ba94c110419409f8f705e6a0"
server_key_iv: "773dd8edeb2c5756c9b1f69e5fc48d78"
server_key_version: "encryptionkey@0"
I20260807 18:42:58.114071 9058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "47b7db9766544b76a77f946b79233a1f"
format_stamp: "Formatted at 2026-08-07 18:42:58 on dist-test-slave-2d19"
server_key: "c3fbcbe2ba94c110419409f8f705e6a0"
server_key_iv: "773dd8edeb2c5756c9b1f69e5fc48d78"
server_key_version: "encryptionkey@0"
I20260807 18:42:58.118280 9058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:42:58.121069 9082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:58.122397 9058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:42:58.122581 9058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "47b7db9766544b76a77f946b79233a1f"
format_stamp: "Formatted at 2026-08-07 18:42:58 on dist-test-slave-2d19"
server_key: "c3fbcbe2ba94c110419409f8f705e6a0"
server_key_iv: "773dd8edeb2c5756c9b1f69e5fc48d78"
server_key_version: "encryptionkey@0"
I20260807 18:42:58.122709 9058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:58.132531 9058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:58.135666 9058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:58.136040 9058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:58.143677 9058 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:40471
I20260807 18:42:58.143707 9165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:40471 every 8 connection(s)
I20260807 18:42:58.144795 9058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:58.147748 9166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:58.148551 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 9058
I20260807 18:42:58.148710 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:58.149039 6697 external_mini_cluster.cc:1428] Setting key e9d1e1c890beeb3a6bbe23d2dd2fcc8a
I20260807 18:42:58.154515 9166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f: Bootstrap starting.
I20260807 18:42:58.156945 9166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:58.157745 9166 log.cc:824] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:58.158830 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:58.150649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41931 (local address 127.6.138.126:40471)
0807 18:42:58.151166 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:58.151182 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:58.151221 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:42:58.151904 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:58.151924 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:58.152375 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:58.152740 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:58.153177 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:58.154170 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:58.155172 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:58.155689 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:58.155875 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:58.156836 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:58.156847 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:58.156861 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:58.157321 (+ 460us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:58.157381 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:58.157406 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:58.157520 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:58.157643 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 18:42:58.157893 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":151,"threads_started":1}
I20260807 18:42:58.159955 9166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f: No bootstrap required, opened a new log
I20260807 18:42:58.162048 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:58.164338 9166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b7db9766544b76a77f946b79233a1f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40471 } }
I20260807 18:42:58.164669 9166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:58.164728 9166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b7db9766544b76a77f946b79233a1f, State: Initialized, Role: FOLLOWER
I20260807 18:42:58.165292 9166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b7db9766544b76a77f946b79233a1f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40471 } }
I20260807 18:42:58.165441 9166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:58.165509 9166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:58.165635 9166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:58.166894 9166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b7db9766544b76a77f946b79233a1f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40471 } }
I20260807 18:42:58.167274 9166 leader_election.cc:302] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47b7db9766544b76a77f946b79233a1f; no voters:
I20260807 18:42:58.167645 9166 leader_election.cc:288] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:58.167757 9182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:58.167989 9182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [term 1 LEADER]: Becoming Leader. State: Replica: 47b7db9766544b76a77f946b79233a1f, State: Running, Role: LEADER
I20260807 18:42:58.168412 9182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b7db9766544b76a77f946b79233a1f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40471 } }
I20260807 18:42:58.169063 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:58.164130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42655 (local address 127.6.138.126:40471)
0807 18:42:58.164290 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:58.164294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:58.164510 (+ 216us) server_negotiation.cc:407] Connection header received
0807 18:42:58.164774 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:58.164777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:58.164850 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:58.164944 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:58.165267 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:58.165918 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:58.167078 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:42:58.167325 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:42:58.167451 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:42:58.168422 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:58.168426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:58.168429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:58.168442 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:58.168473 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:58.168486 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:58.168549 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:58.168887 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 18:42:58.168941 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:42:58.169178 9166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:58.170565 9184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47b7db9766544b76a77f946b79233a1f. Latest consensus state: current_term: 1 leader_uuid: "47b7db9766544b76a77f946b79233a1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b7db9766544b76a77f946b79233a1f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40471 } } }
I20260807 18:42:58.170677 9184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:58.171084 9195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:58.171684 9183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47b7db9766544b76a77f946b79233a1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b7db9766544b76a77f946b79233a1f" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40471 } } }
I20260807 18:42:58.172137 9183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:58.174672 9195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:58.180898 9195 catalog_manager.cc:1347] Generated new cluster ID: e5e4706cfe05403580bb5e8a8754e902
I20260807 18:42:58.180995 9195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:58.195037 9195 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:58.196090 9195 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:58.203984 9195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47b7db9766544b76a77f946b79233a1f: Generated new TSK 0
I20260807 18:42:58.204748 9195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:58.213100 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 9058
2026-08-07T18:42:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1658 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:58 dist-test-slave-2d19 krb5kdc[9214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:58 dist-test-slave-2d19 krb5kdc[9214](info): set up 2 sockets
Aug 07 18:42:58 dist-test-slave-2d19 krb5kdc[9214](info): commencing operation
krb5kdc: starting...
W20260807 18:42:59.477193 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:42:59 dist-test-slave-2d19 krb5kdc[9214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:42:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:42:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:42:59.612138 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:40563
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:40563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:42:59.728744 9237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:42:59.729061 9237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:42:59.729139 9237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:42:59.733207 9237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:42:59.733295 9237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:42:59.733333 9237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:42:59.733393 9237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:42:59.733429 9237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:42:59.738785 9237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:40563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:40563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:42:59.740103 9237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:42:59.741169 9237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:42:59.749728 9244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:59.749754 9245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:42:59.749728 9247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:42:59.750178 9237 server_base.cc:1034] running on GCE node
I20260807 18:42:59.750783 9237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:42:59.751914 9237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:42:59.753100 9237 hybrid_clock.cc:648] HybridClock initialized: now 1786128179753056 us; error 52 us; skew 500 ppm
Aug 07 18:42:59 dist-test-slave-2d19 krb5kdc[9214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128179, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:42:59.756028 9237 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:42:59.756981 9237 webserver.cc:460] Webserver started at http://127.6.138.126:35937/ using document root <none> and password file <none>
I20260807 18:42:59.757567 9237 fs_manager.cc:359] Metadata directory not provided
I20260807 18:42:59.757669 9237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:42:59.757915 9237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:42:59.760036 9237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "09de0b3b04e94491bc65f26ec8aa8d06"
format_stamp: "Formatted at 2026-08-07 18:42:59 on dist-test-slave-2d19"
server_key: "88899e2e1a6bdb163849dc9003644eb5"
server_key_iv: "7c1f972c5ea3dfd7d04282e56c350901"
server_key_version: "encryptionkey@0"
I20260807 18:42:59.760656 9237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "09de0b3b04e94491bc65f26ec8aa8d06"
format_stamp: "Formatted at 2026-08-07 18:42:59 on dist-test-slave-2d19"
server_key: "88899e2e1a6bdb163849dc9003644eb5"
server_key_iv: "7c1f972c5ea3dfd7d04282e56c350901"
server_key_version: "encryptionkey@0"
I20260807 18:42:59.764463 9237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:42:59.766811 9254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:42:59.768049 9237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:42:59.768201 9237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "09de0b3b04e94491bc65f26ec8aa8d06"
format_stamp: "Formatted at 2026-08-07 18:42:59 on dist-test-slave-2d19"
server_key: "88899e2e1a6bdb163849dc9003644eb5"
server_key_iv: "7c1f972c5ea3dfd7d04282e56c350901"
server_key_version: "encryptionkey@0"
I20260807 18:42:59.768358 9237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:42:59.772922 9237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:42:59.782753 9237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:42:59.782969 9237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:42:59.790170 9237 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:40563
I20260807 18:42:59.790192 9342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:40563 every 8 connection(s)
I20260807 18:42:59.791126 9237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:42:59.793952 9343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:42:59.799633 9343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06: Bootstrap starting.
I20260807 18:42:59.799817 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 9237
I20260807 18:42:59.799948 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:42:59.800298 6697 external_mini_cluster.cc:1428] Setting key a2a3b4043041f13c1263f6ba294e649f
I20260807 18:42:59.802433 9343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06: Neither blocks nor log segments found. Creating new log.
I20260807 18:42:59.803195 9343 log.cc:824] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06: Log is configured to *not* fsync() on all Append() calls
I20260807 18:42:59.805570 9343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06: No bootstrap required, opened a new log
I20260807 18:42:59.805929 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:59.801690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56875 (local address 127.6.138.126:40563)
0807 18:42:59.802179 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:59.802194 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:59.802232 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:42:59.802937 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:59.802957 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:59.803255 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:59.803659 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:59.804516 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:59.804521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:59.804536 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:59.804549 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:59.804615 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:59.804640 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:59.804760 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:59.804844 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 18:42:59.805010 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":134,"threads_started":1}
I20260807 18:42:59.808971 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:42:59.809839 9343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09de0b3b04e94491bc65f26ec8aa8d06" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40563 } }
I20260807 18:42:59.810061 9343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:42:59.810122 9343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09de0b3b04e94491bc65f26ec8aa8d06, State: Initialized, Role: FOLLOWER
I20260807 18:42:59.810670 9343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09de0b3b04e94491bc65f26ec8aa8d06" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40563 } }
I20260807 18:42:59.810814 9343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:42:59.810909 9343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:42:59.811048 9343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:42:59.812497 9343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09de0b3b04e94491bc65f26ec8aa8d06" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40563 } }
I20260807 18:42:59.812945 9343 leader_election.cc:302] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09de0b3b04e94491bc65f26ec8aa8d06; no voters:
I20260807 18:42:59.813127 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:42:59.811212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37919 (local address 127.6.138.126:40563)
0807 18:42:59.811364 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:42:59.811369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:42:59.811554 (+ 185us) server_negotiation.cc:407] Connection header received
0807 18:42:59.811651 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:42:59.811654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:42:59.811691 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:42:59.811769 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:42:59.812652 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:42:59.812656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:42:59.812659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:42:59.812671 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:42:59.812702 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:42:59.812714 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:42:59.812804 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:42:59.812947 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 18:42:59.812990 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:42:59.813356 9343 leader_election.cc:288] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:42:59.813428 9358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:42:59.813665 9358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [term 1 LEADER]: Becoming Leader. State: Replica: 09de0b3b04e94491bc65f26ec8aa8d06, State: Running, Role: LEADER
I20260807 18:42:59.814071 9358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09de0b3b04e94491bc65f26ec8aa8d06" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40563 } }
I20260807 18:42:59.814838 9343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:42:59.816082 9359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09de0b3b04e94491bc65f26ec8aa8d06. Latest consensus state: current_term: 1 leader_uuid: "09de0b3b04e94491bc65f26ec8aa8d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09de0b3b04e94491bc65f26ec8aa8d06" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40563 } } }
I20260807 18:42:59.816098 9360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09de0b3b04e94491bc65f26ec8aa8d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09de0b3b04e94491bc65f26ec8aa8d06" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 40563 } } }
I20260807 18:42:59.816282 9360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:59.816282 9359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06 [sys.catalog]: This master's current role is: LEADER
I20260807 18:42:59.816704 9367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:42:59.819962 9367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:42:59.825598 9367 catalog_manager.cc:1347] Generated new cluster ID: 3b19686cb2c1459a9c7c5a0901f6eba8
I20260807 18:42:59.825676 9367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:42:59.832079 9367 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:42:59.832418 9367 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:42:59.844370 9367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09de0b3b04e94491bc65f26ec8aa8d06: Generated new TSK 0
I20260807 18:42:59.845155 9367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:42:59.855950 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 9237
2026-08-07T18:42:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1639 ms)
[----------] 18 tests from AuthTokenIssuingTest (30172 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:42:59 dist-test-slave-2d19 krb5kdc[9391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:42:59 dist-test-slave-2d19 krb5kdc[9391](info): set up 2 sockets
Aug 07 18:42:59 dist-test-slave-2d19 krb5kdc[9391](info): commencing operation
krb5kdc: starting...
W20260807 18:43:01.165750 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:43:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:43:01.308187 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42319
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42319,127.6.138.125:39487,127.6.138.124:39473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:43:01.426048 9409 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:43:01.426429 9409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:43:01.426504 9409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:43:01.426584 9409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:43:01.430639 9409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:43:01.430740 9409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:43:01.430783 9409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:43:01.430820 9409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:43:01.430856 9409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:43:01.436314 9409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42319,127.6.138.125:39487,127.6.138.124:39473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:42319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:43:01.437492 9409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:43:01.438572 9409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:43:01.447449 9426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:43:01.447527 9409 server_base.cc:1034] running on GCE node
W20260807 18:43:01.447436 9421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:43:01.447427 9418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:43:01.448417 9409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:43:01.449512 9409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:43:01.450683 9409 hybrid_clock.cc:648] HybridClock initialized: now 1786128181450663 us; error 32 us; skew 500 ppm
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:43:01.453550 9409 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:43:01.454465 9409 webserver.cc:460] Webserver started at http://127.6.138.126:34175/ using document root <none> and password file <none>
I20260807 18:43:01.455000 9409 fs_manager.cc:359] Metadata directory not provided
I20260807 18:43:01.455061 9409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:43:01.455232 9409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:43:01.457428 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "e6ac9a4845974d62a9e63befe642b24a"
format_stamp: "Formatted at 2026-08-07 18:43:01 on dist-test-slave-2d19"
server_key: "d105a9037522d46b2658e1dfb6bbb515"
server_key_iv: "a4227e65cbd62716c540252d9cfea511"
server_key_version: "encryptionkey@0"
I20260807 18:43:01.458010 9409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "e6ac9a4845974d62a9e63befe642b24a"
format_stamp: "Formatted at 2026-08-07 18:43:01 on dist-test-slave-2d19"
server_key: "d105a9037522d46b2658e1dfb6bbb515"
server_key_iv: "a4227e65cbd62716c540252d9cfea511"
server_key_version: "encryptionkey@0"
I20260807 18:43:01.461855 9409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:43:01.464341 9435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:43:01.465822 9409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:43:01.466017 9409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "e6ac9a4845974d62a9e63befe642b24a"
format_stamp: "Formatted at 2026-08-07 18:43:01 on dist-test-slave-2d19"
server_key: "d105a9037522d46b2658e1dfb6bbb515"
server_key_iv: "a4227e65cbd62716c540252d9cfea511"
server_key_version: "encryptionkey@0"
I20260807 18:43:01.466146 9409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:43:01.485852 9409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:43:01.489310 9409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:43:01.489555 9409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:43:01.496781 9409 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:42319
I20260807 18:43:01.497108 9522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:42319 every 8 connection(s)
I20260807 18:43:01.497794 9409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:43:01.500735 9523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:43:01.504541 9523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:01.506290 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 9409
I20260807 18:43:01.506449 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:43:01.506740 6697 external_mini_cluster.cc:1428] Setting key fb2f83295f08fe410c72cbf59c919f3f
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.125@KRBTEST.COM" created.
I20260807 18:43:01.521531 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.507764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.6.138.126:42319)
0807 18:43:01.508035 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:01.508044 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:01.508088 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:43:01.509296 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:01.509305 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:01.509563 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:01.509782 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:01.510579 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.512017 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.513211 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.513922 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.514138 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.516581 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:01.516592 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:01.516608 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:01.516641 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:01.518348 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.518850 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.518855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.518857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.518986 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.519340 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.519343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.519345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.519767 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:01.519896 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:01.520415 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 18:43:01.520720 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
I20260807 18:43:01.521540 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.506956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42319 (local address 127.0.0.1:38590)
0807 18:43:01.507544 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.507608 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:01.508914 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:01.509793 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:01.509798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:01.510151 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:01.510414 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.510427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.512185 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.512188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.513075 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.513086 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.514106 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.514109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.514189 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.514871 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:01.514888 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:01.516418 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:01.518459 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.518468 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.518477 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.518710 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.519086 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.519089 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.519090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.519206 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.519893 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:01.519899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:01.520038 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 18:43:01.520184 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:43:01.520630 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":54,"thread_start_us":145,"threads_started":1}
I20260807 18:43:01.524437 9523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:01.525270 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.524923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:39487 (local address 127.0.0.1:51718)
0807 18:43:01.525083 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.525179 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:43:01.525519 9440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
I20260807 18:43:01.526496 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.526222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:39487 (local address 127.0.0.1:51734)
0807 18:43:01.526355 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.526425 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:43:01.526676 9523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.125:39487: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab.
Entry for principal kudu/127.6.138.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab.
I20260807 18:43:01.539320 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.125:39487
--webserver_interface=127.6.138.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab
--principal=kudu/127.6.138.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42319,127.6.138.125:39487,127.6.138.124:39473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:43:01.575965 9523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } attempt: 1
I20260807 18:43:01.576972 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.576597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:39487 (local address 127.0.0.1:51736)
0807 18:43:01.576782 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.576871 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 18:43:01.577782 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.577535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:39487 (local address 127.0.0.1:51748)
0807 18:43:01.577660 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.577716 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:43:01.577962 9523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.125:39487: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
I20260807 18:43:01.646173 9523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } attempt: 2
I20260807 18:43:01.647209 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.646885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:39487 (local address 127.0.0.1:51762)
0807 18:43:01.647046 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.647129 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:43:01.648072 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.647856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:39487 (local address 127.0.0.1:51774)
0807 18:43:01.647968 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.648013 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 18:43:01.648247 9523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.125:39487: Network error: Client connection negotiation failed: client connection to 127.6.138.125:39487: connect: Connection refused (error 111)
W20260807 18:43:01.656659 9533 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:43:01.656998 9533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:43:01.657034 9533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:43:01.657081 9533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:43:01.661070 9533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:43:01.661125 9533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:43:01.661144 9533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:43:01.661161 9533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:43:01.661177 9533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:43:01.666715 9533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42319,127.6.138.125:39487,127.6.138.124:39473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab
--principal=kudu/127.6.138.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.125:39487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.138.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:43:01.667797 9533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:43:01.668886 9533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:43:01.678122 9542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:43:01.678109 9540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:43:01.678103 9533 server_base.cc:1034] running on GCE node
W20260807 18:43:01.678098 9544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:43:01.678985 9533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:43:01.680145 9533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:43:01.681344 9533 hybrid_clock.cc:648] HybridClock initialized: now 1786128181681321 us; error 33 us; skew 500 ppm
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:43:01.684224 9533 init.cc:377] Logged in from keytab as kudu/127.6.138.125@KRBTEST.COM (short username kudu)
I20260807 18:43:01.685230 9533 webserver.cc:460] Webserver started at http://127.6.138.125:34679/ using document root <none> and password file <none>
I20260807 18:43:01.685828 9533 fs_manager.cc:359] Metadata directory not provided
I20260807 18:43:01.685923 9533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:43:01.686177 9533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:43:01.688396 9533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/data/instance:
uuid: "e102f711aa454cdda17da1efc9d8c69c"
format_stamp: "Formatted at 2026-08-07 18:43:01 on dist-test-slave-2d19"
server_key: "97a071189eeb2648cbca6129bb7f163e"
server_key_iv: "b5fd40b77e020e4c554c56257c57b6d8"
server_key_version: "encryptionkey@0"
I20260807 18:43:01.688992 9533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/wal/instance:
uuid: "e102f711aa454cdda17da1efc9d8c69c"
format_stamp: "Formatted at 2026-08-07 18:43:01 on dist-test-slave-2d19"
server_key: "97a071189eeb2648cbca6129bb7f163e"
server_key_iv: "b5fd40b77e020e4c554c56257c57b6d8"
server_key_version: "encryptionkey@0"
I20260807 18:43:01.692899 9533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:43:01.695433 9551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:43:01.696862 9533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:43:01.697008 9533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/wal
uuid: "e102f711aa454cdda17da1efc9d8c69c"
format_stamp: "Formatted at 2026-08-07 18:43:01 on dist-test-slave-2d19"
server_key: "97a071189eeb2648cbca6129bb7f163e"
server_key_iv: "b5fd40b77e020e4c554c56257c57b6d8"
server_key_version: "encryptionkey@0"
I20260807 18:43:01.697131 9533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:43:01.718920 9533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:43:01.722012 9533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:43:01.722249 9533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:43:01.729830 9533 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.125:39487
I20260807 18:43:01.730145 9636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.125:39487 every 8 connection(s)
I20260807 18:43:01.730875 9533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
I20260807 18:43:01.734112 9637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:43:01.737404 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 9533
I20260807 18:43:01.737524 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-1/wal/instance
I20260807 18:43:01.737843 6697 external_mini_cluster.cc:1428] Setting key bd8a5b32b4c10c62e1e04b0391553c14
I20260807 18:43:01.738076 9637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:01.739459 9523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } attempt: 3
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.138.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.124@KRBTEST.COM" created.
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:43:01.754438 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.740558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42319 (local address 127.0.0.1:38600)
0807 18:43:01.741209 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.741258 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:01.742446 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:01.742930 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:01.742940 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:01.743307 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:01.744017 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.744032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.744866 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.744871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.745847 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.745858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.746583 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.746587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.746775 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.747452 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:01.747468 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:01.749214 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:01.750879 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.750885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.750898 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.751188 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.751464 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.751466 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.751468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.751611 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.752130 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:01.752137 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:01.752543 (+ 406us) client_negotiation.cc:769] Sending connection context
0807 18:43:01.752787 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 18:43:01.753167 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":121,"spinlock_wait_cycles":28928,"thread_start_us":165,"threads_started":1}
I20260807 18:43:01.754702 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.740192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.6.138.126:42319)
0807 18:43:01.740344 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:01.740348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:01.741349 (+ 1001us) server_negotiation.cc:407] Connection header received
0807 18:43:01.742703 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:01.742707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:01.742774 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:01.742869 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:01.744163 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.744753 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.745994 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.746419 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.746543 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.749359 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:01.749362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:01.749364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:01.749388 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:01.750766 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.751294 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.751297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.751298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.751377 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.751753 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.751759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.751764 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.751972 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:01.752102 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:01.754379 (+ 2277us) server_negotiation.cc:299] Negotiation successful
0807 18:43:01.754528 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:43:01.756330 9637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:01.757506 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.739988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:39487 (local address 127.0.0.1:51782)
0807 18:43:01.740137 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.740155 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:01.740294 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:01.742925 (+ 2631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:01.742931 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:01.743173 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:01.743329 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.743343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.745575 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.745579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.746244 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.746254 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.747040 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.747410 (+ 370us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.747545 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.748067 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:01.748088 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:01.752592 (+ 4504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:01.755708 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.755712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.755714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.755928 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.756313 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.756318 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.756321 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.756433 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.757026 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:01.757030 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:01.757140 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 18:43:01.757226 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 18:43:01.757378 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 18:43:01.758095 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.740558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51782 (local address 127.6.138.125:39487)
0807 18:43:01.741239 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:01.741259 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:01.741302 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:43:01.742427 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:01.742454 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:01.742721 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:01.743548 (+ 827us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:01.743810 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.745422 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.746380 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.746884 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.747011 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.752748 (+ 5737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:01.752762 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:01.752779 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:01.752804 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:01.755539 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.756068 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.756074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.756078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.756176 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.756546 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.756551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.756552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.756891 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:01.757733 (+ 842us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:01.757848 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 18:43:01.757965 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":528,"thread_start_us":150,"threads_started":1}
I20260807 18:43:01.760001 9523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:01.760848 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.760559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50650)
0807 18:43:01.760742 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.760790 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 18:43:01.761641 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.761397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50660)
0807 18:43:01.761537 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.761590 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 18:43:01.761853 9523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39473: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:43:01.769984 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.756953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:39487 (local address 127.0.0.1:51796)
0807 18:43:01.757159 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.757174 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:01.757250 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:01.759253 (+ 2003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:01.759260 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:01.759478 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:01.759618 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.759629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.760690 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.760697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.761611 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.761623 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.762301 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.762305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.762417 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.762895 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:01.762909 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:01.766567 (+ 3658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:01.768303 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.768308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.768310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.768661 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.768939 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.768942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.768944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.769056 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.769438 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:01.769457 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:01.769561 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 18:43:01.769669 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:43:01.769815 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260807 18:43:01.770221 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.757033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51796 (local address 127.6.138.125:39487)
0807 18:43:01.759014 (+ 1981us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:01.759018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:01.759031 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:43:01.759064 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:01.759067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:01.759107 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:01.759191 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:01.759762 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.760568 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.761889 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.762173 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.762257 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.766708 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:01.766711 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:01.766713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:01.766736 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:01.768148 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.768755 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.768757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.768759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.768846 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.769152 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.769155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.769156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.769337 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:01.769415 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:01.769746 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 18:43:01.769875 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":1922,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab.
Entry for principal kudu/127.6.138.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab.
I20260807 18:43:01.770900 9637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:01.771662 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.771390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50662)
0807 18:43:01.771519 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.771613 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 18:43:01.772027 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.124:39473
--webserver_interface=127.6.138.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab
--principal=kudu/127.6.138.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:42319,127.6.138.125:39487,127.6.138.124:39473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:43:01.772475 9555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
I20260807 18:43:01.773461 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.773223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50676)
0807 18:43:01.773363 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.773411 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 18:43:01.773684 9637 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39473: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
I20260807 18:43:01.793040 9523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } attempt: 1
I20260807 18:43:01.793915 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.793613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50684)
0807 18:43:01.793766 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.793836 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:43:01.794668 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.794433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50700)
0807 18:43:01.794574 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.794616 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 18:43:01.794802 9523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39473: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
I20260807 18:43:01.822993 9637 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } attempt: 1
I20260807 18:43:01.823967 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.823658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50714)
0807 18:43:01.823823 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.823887 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:43:01.824795 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.824566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50716)
0807 18:43:01.824686 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.824739 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 18:43:01.824923 9637 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39473: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
I20260807 18:43:01.869957 9523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } attempt: 2
I20260807 18:43:01.870862 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.870567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50722)
0807 18:43:01.870699 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.870763 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 18:43:01.872406 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.872034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50732)
0807 18:43:01.872232 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.872319 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260807 18:43:01.872664 9523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39473: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
I20260807 18:43:01.893076 9637 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } attempt: 2
I20260807 18:43:01.894105 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.893676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50748)
0807 18:43:01.893944 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.894009 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260807 18:43:01.895023 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.894763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50756)
0807 18:43:01.894905 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.894952 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 18:43:01.895237 9637 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39473: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
W20260807 18:43:01.909614 9648 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:43:01.910125 9648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:43:01.910197 9648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:43:01.910287 9648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:43:01.919193 9648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:43:01.919304 9648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:43:01.919348 9648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:43:01.919394 9648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:43:01.919435 9648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:43:01.925769 9648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:42319,127.6.138.125:39487,127.6.138.124:39473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab
--principal=kudu/127.6.138.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.124:39473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.138.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:43:01.927129 9648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:43:01.928228 9648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:43:01.937144 9656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:43:01.937167 9659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:43:01.937146 9655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:43:01.937278 9648 server_base.cc:1034] running on GCE node
I20260807 18:43:01.938055 9648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:43:01.939208 9648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:43:01.940395 9648 hybrid_clock.cc:648] HybridClock initialized: now 1786128181940362 us; error 55 us; skew 500 ppm
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:43:01.943238 9648 init.cc:377] Logged in from keytab as kudu/127.6.138.124@KRBTEST.COM (short username kudu)
I20260807 18:43:01.944087 9648 webserver.cc:460] Webserver started at http://127.6.138.124:41779/ using document root <none> and password file <none>
I20260807 18:43:01.944718 9648 fs_manager.cc:359] Metadata directory not provided
I20260807 18:43:01.944782 9648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:43:01.945024 9648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:43:01.947134 9648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/data/instance:
uuid: "08e7a5987293429ab83010e2ccda8669"
format_stamp: "Formatted at 2026-08-07 18:43:01 on dist-test-slave-2d19"
server_key: "cfd1eb8aa9450f0ca0b2a1553ec4496b"
server_key_iv: "69d8b2a634ddcc5fc76a059110516d00"
server_key_version: "encryptionkey@0"
I20260807 18:43:01.947721 9648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/wal/instance:
uuid: "08e7a5987293429ab83010e2ccda8669"
format_stamp: "Formatted at 2026-08-07 18:43:01 on dist-test-slave-2d19"
server_key: "cfd1eb8aa9450f0ca0b2a1553ec4496b"
server_key_iv: "69d8b2a634ddcc5fc76a059110516d00"
server_key_version: "encryptionkey@0"
I20260807 18:43:01.951614 9648 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:43:01.954157 9671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:43:01.955422 9648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:43:01.955572 9648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/wal
uuid: "08e7a5987293429ab83010e2ccda8669"
format_stamp: "Formatted at 2026-08-07 18:43:01 on dist-test-slave-2d19"
server_key: "cfd1eb8aa9450f0ca0b2a1553ec4496b"
server_key_iv: "69d8b2a634ddcc5fc76a059110516d00"
server_key_version: "encryptionkey@0"
I20260807 18:43:01.955688 9648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:43:01.971833 9523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } attempt: 3
I20260807 18:43:01.972726 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.972441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50764)
0807 18:43:01.972579 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.972647 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 18:43:01.973506 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.973291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50766)
0807 18:43:01.973401 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.973445 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:43:01.973690 9523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39473: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39473: connect: Connection refused (error 111)
I20260807 18:43:01.974521 9648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:43:01.978046 9648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:43:01.978266 9648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:43:01.985481 9648 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.124:39473
I20260807 18:43:01.985782 9756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.124:39473 every 8 connection(s)
I20260807 18:43:01.986406 9637 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } attempt: 3
I20260807 18:43:01.986588 9648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
I20260807 18:43:01.989953 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:43:01.992677 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 9648
I20260807 18:43:01.992830 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127881318083-6697-0/minicluster-data/master-2/wal/instance
I20260807 18:43:01.993151 6697 external_mini_cluster.cc:1428] Setting key e5fbc1a0836f25268a988b7f14ee6341
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
I20260807 18:43:01.995325 9757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:01.999953 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.986988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50768)
0807 18:43:01.987165 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.987188 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:01.987277 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:01.989588 (+ 2311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:01.989594 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:01.989796 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:01.989916 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.989924 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.991367 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.991374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.992183 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.992197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.993108 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.993112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:01.993194 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.993756 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:01.993770 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:01.995361 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:01.997872 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.997875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.997877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.998152 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.998540 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:01.998544 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:01.998546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:01.998644 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:01.999415 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:01.999419 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:01.999523 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 18:43:01.999643 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 18:43:01.999787 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 18:43:02.001199 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.987624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50768 (local address 127.6.138.124:39473)
0807 18:43:01.988070 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:01.988098 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:01.988135 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:43:01.988913 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:01.988934 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:01.989278 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:01.989573 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:01.990105 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.991206 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.992369 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.992945 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.993159 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:01.995560 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:01.995565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:01.995578 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:01.995604 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:01.997731 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.998273 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.998280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.998286 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.998437 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:01.998763 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:01.998766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:01.998767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:01.999277 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:01.999445 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:01.999809 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.000395 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":148,"threads_started":1}
Aug 07 18:43:01 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
Aug 07 18:43:02 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:43:02.005682 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.994456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.6.138.124:39473)
0807 18:43:01.994793 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:01.994803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:01.994816 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:43:01.994851 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:01.994854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:01.994895 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:01.994989 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:01.995947 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.996742 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.997991 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.998353 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.998514 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.002010 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.002013 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.002015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.002036 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.003713 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.004131 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.004134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.004136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.004233 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.004659 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.004662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.004664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.004897 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.005083 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.005366 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.005483 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":100,"threads_started":1}
I20260807 18:43:02.006114 9637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c: Bootstrap starting.
I20260807 18:43:02.009227 9637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c: Neither blocks nor log segments found. Creating new log.
I20260807 18:43:02.009680 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.997372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:42319 (local address 127.0.0.1:38604)
0807 18:43:01.997804 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:01.997853 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:01.998016 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:01.998356 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:01.998362 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:01.998813 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:01.998968 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:01.998974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.999712 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:01.999717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:02.000821 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:02.000836 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.001596 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.001601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:02.001749 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.002302 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:02.002319 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:02.005846 (+ 3527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:02.007773 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:02.007782 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:02.007786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:02.008039 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:02.008365 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:02.008369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:02.008370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:02.008466 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:02.008948 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:02.008953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:02.009078 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 18:43:02.009247 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 18:43:02.009497 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":102,"threads_started":1}
I20260807 18:43:02.010170 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:01.997422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.6.138.126:42319)
0807 18:43:01.997550 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:01.997555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:01.997930 (+ 375us) server_negotiation.cc:407] Connection header received
0807 18:43:01.998181 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:01.998184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:01.998230 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:01.998323 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:01.999108 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:01.999595 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.000962 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.001360 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.001474 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.006013 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.006017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.006019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.006051 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.007657 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.008145 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.008149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.008151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.008242 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.008576 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.008579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.008581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.008817 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.008915 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.009309 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.009994 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:43:02.010478 9637 log.cc:824] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:43:02.011188 9757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:02.013053 9637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c: No bootstrap required, opened a new log
Aug 07 18:43:02 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
Aug 07 18:43:02 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:43:02.017443 9637 raft_consensus.cc:348] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } }
I20260807 18:43:02.017709 9637 raft_consensus.cc:729] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e102f711aa454cdda17da1efc9d8c69c, State: Initialized, Role: FOLLOWER
I20260807 18:43:02.018083 9637 consensus_queue.cc:260] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } }
I20260807 18:43:02.019260 9768 sys_catalog.cc:455] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.019416 9768 sys_catalog.cc:458] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:02.019685 9637 sys_catalog.cc:564] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:43:02.021216 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.011785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51798 (local address 127.6.138.125:39487)
0807 18:43:02.011906 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.011909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.012049 (+ 140us) server_negotiation.cc:407] Connection header received
0807 18:43:02.012161 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.012164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.012204 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.012351 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.012828 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.013512 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.014477 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.014808 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.014923 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.017167 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.017171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.017174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.017199 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.018957 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.019608 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.019611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.019613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.019705 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.020048 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.020052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.020068 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.020367 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.020447 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.020889 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.021034 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:43:02.021802 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.011702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:39487 (local address 127.0.0.1:51798)
0807 18:43:02.011975 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:02.011985 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:02.012084 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:02.012362 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:02.012364 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:02.012568 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:02.012701 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:02.012708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.013620 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.013623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:02.014351 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:02.014362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.014998 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.015001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:02.015083 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.015606 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:02.015627 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:02.017012 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:02.019071 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:02.019076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:02.019078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:02.019344 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:02.019831 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:02.019834 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:02.019836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:02.019930 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:02.020535 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:02.020539 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:02.020653 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 18:43:02.021410 (+ 757us) client_negotiation.cc:240] Negotiation successful
0807 18:43:02.021591 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219}
I20260807 18:43:02.021932 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.009656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38618 (local address 127.6.138.126:42319)
0807 18:43:02.010556 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.010569 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.010586 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:43:02.010625 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.010631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.010682 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.010766 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.011662 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.012523 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.013534 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.013877 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.013984 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.017943 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.017946 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.017948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.017973 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.019773 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.020206 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.020209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.020211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.020328 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.020672 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.020676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.020677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.020970 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.021187 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.021560 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.021716 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":140,"threads_started":1}
I20260807 18:43:02.022552 9757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } has no permanent_uuid. Determining permanent_uuid...
W20260807 18:43:02.027165 9790 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:43:02.027252 9790 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 18:43:02 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
Aug 07 18:43:02 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:43:02.025017 9533 master_runner.cc:186] Error getting master registration for 127.6.138.126:42319: OK, instance_id { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" instance_seqno: 1786128181494807 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:43:02.031131 9533 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:43:02.032984 9765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.023002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50788)
0807 18:43:02.023110 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:02.023127 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:02.023208 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:02.023534 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:02.023536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:02.023701 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:02.023789 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:02.023796 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.024799 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.024802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:02.025538 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:02.025550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.026181 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.026185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:02.026250 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.026719 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:02.026734 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:02.028177 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:02.030245 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:02.030247 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:02.030249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:02.030497 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:02.031565 (+ 1068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:02.031570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:02.031572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:02.031684 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:02.032342 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:02.032348 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:02.032454 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 18:43:02.032593 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 18:43:02.032737 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 18:43:02.033046 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.023195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.6.138.124:39473)
0807 18:43:02.023335 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.023341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.023355 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:43:02.023391 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.023394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.023439 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.023528 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.023916 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.024677 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.025675 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.025986 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.026080 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.028339 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.028344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.028348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.028372 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.030114 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.030728 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.030734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.030738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.030837 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.031811 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.031815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.031817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.032141 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.032286 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.032672 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.032812 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:43:02.034439 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.023437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51806 (local address 127.6.138.125:39487)
0807 18:43:02.023545 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.023549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.023560 (+ 11us) server_negotiation.cc:407] Connection header received
0807 18:43:02.023598 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.023599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.023630 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.023723 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.024836 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.025568 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.026663 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.027028 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.027162 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.030254 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.030257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.030259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.030287 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.032135 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.032657 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.032661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.032662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.032747 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.033075 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.033080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.033084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.033293 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.033512 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.034143 (+ 631us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.034255 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:43:02.036002 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669: Bootstrap starting.
I20260807 18:43:02.039206 9757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669: Neither blocks nor log segments found. Creating new log.
I20260807 18:43:02.040122 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:43:02.040087 9757 log.cc:824] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669: Log is configured to *not* fsync() on all Append() calls
I20260807 18:43:02.042284 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669: No bootstrap required, opened a new log
I20260807 18:43:02.041604 9474 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e6ac9a4845974d62a9e63befe642b24a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38618
I20260807 18:43:02.045519 9757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } }
I20260807 18:43:02.045722 9757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e7a5987293429ab83010e2ccda8669, State: Initialized, Role: FOLLOWER
I20260807 18:43:02.045985 9757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } }
I20260807 18:43:02.046567 9793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.046906 9793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:02.047132 9757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:43:02.052716 9815 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:43:02.052795 9815 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:43:02.051270 9648 master_runner.cc:186] Error getting master registration for 127.6.138.126:42319: OK, instance_id { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" instance_seqno: 1786128181494807 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:43:02.055027 9648 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:43:02.137844 9523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } attempt: 4
Aug 07 18:43:02 dist-test-slave-2d19 krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
I20260807 18:43:02.148737 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.138477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39473 (local address 127.0.0.1:50794)
0807 18:43:02.138697 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:02.138712 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:02.138817 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:02.139142 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:02.139145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:02.139369 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:02.139483 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:02.139490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.140816 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.140820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:02.141697 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:02.141711 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.142352 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.142356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:02.142457 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.143072 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:02.143089 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:02.144813 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:02.146861 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:02.146865 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:02.146867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:02.147211 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:02.147510 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:02.147513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:02.147515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:02.147617 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:02.148088 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:02.148091 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:02.148243 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 18:43:02.148399 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 18:43:02.148551 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260807 18:43:02.148772 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.138586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50794 (local address 127.6.138.124:39473)
0807 18:43:02.138719 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.138727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.138793 (+ 66us) server_negotiation.cc:407] Connection header received
0807 18:43:02.138928 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.138932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.138988 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.139094 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.139594 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.140659 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.141868 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.142226 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.142342 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.144982 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.144986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.144989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.145019 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.146750 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.147308 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.147312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.147313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.147415 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.147718 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.147721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.147723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.147946 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.148045 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.148484 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.148609 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:43:02.152590 9523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a: Bootstrap starting.
I20260807 18:43:02.155501 9523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a: Neither blocks nor log segments found. Creating new log.
I20260807 18:43:02.156456 9523 log.cc:824] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:43:02.158794 9523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a: No bootstrap required, opened a new log
I20260807 18:43:02.162194 9523 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } }
I20260807 18:43:02.162494 9523 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ac9a4845974d62a9e63befe642b24a, State: Initialized, Role: FOLLOWER
I20260807 18:43:02.162873 9523 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } }
I20260807 18:43:02.164335 9523 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:43:02.164486 9474 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:43:02.164664 9474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:43:02.166150 9474 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } }
I20260807 18:43:02.167215 9820 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.167341 9820 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:02.168612 9591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e6ac9a4845974d62a9e63befe642b24a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e102f711aa454cdda17da1efc9d8c69c"
I20260807 18:43:02.168884 9591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:43:02.170353 9591 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ac9a4845974d62a9e63befe642b24a in term 1.
I20260807 18:43:02.170858 9440 leader_election.cc:302] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e102f711aa454cdda17da1efc9d8c69c, e6ac9a4845974d62a9e63befe642b24a; no voters:
I20260807 18:43:02.171483 9474 leader_election.cc:288] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [CANDIDATE]: Term 1 election: Requested vote from peers e102f711aa454cdda17da1efc9d8c69c (127.6.138.125:39487), 08e7a5987293429ab83010e2ccda8669 (127.6.138.124:39473)
I20260807 18:43:02.171686 9820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:43:02.171929 9820 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [term 1 LEADER]: Becoming Leader. State: Replica: e6ac9a4845974d62a9e63befe642b24a, State: Running, Role: LEADER
I20260807 18:43:02.172325 9820 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } }
I20260807 18:43:02.174255 9719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e6ac9a4845974d62a9e63befe642b24a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "08e7a5987293429ab83010e2ccda8669"
I20260807 18:43:02.174332 9820 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6ac9a4845974d62a9e63befe642b24a. Latest consensus state: current_term: 1 leader_uuid: "e6ac9a4845974d62a9e63befe642b24a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.174460 9820 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [sys.catalog]: This master's current role is: LEADER
I20260807 18:43:02.174504 9719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:43:02.174829 9849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:43:02.178228 9849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:43:02.180400 9719 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ac9a4845974d62a9e63befe642b24a in term 1.
I20260807 18:43:02.183625 9409 master_runner.cc:263] No leader master found from master e6ac9a4845974d62a9e63befe642b24a
I20260807 18:43:02.183701 9409 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:43:02.194320 9719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [term 1 FOLLOWER]: Refusing update from remote peer e6ac9a4845974d62a9e63befe642b24a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:43:02.195216 9591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [term 1 FOLLOWER]: Refusing update from remote peer e6ac9a4845974d62a9e63befe642b24a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:43:02.196132 9820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:43:02.196657 9820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [LEADER]: Connected to new peer: Peer: permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:43:02.204070 9793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6ac9a4845974d62a9e63befe642b24a. Latest consensus state: current_term: 1 leader_uuid: "e6ac9a4845974d62a9e63befe642b24a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.204227 9793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:02.205647 9768 sys_catalog.cc:455] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6ac9a4845974d62a9e63befe642b24a. Latest consensus state: current_term: 1 leader_uuid: "e6ac9a4845974d62a9e63befe642b24a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.205804 9768 sys_catalog.cc:458] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:02.213640 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.185351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51814 (local address 127.6.138.125:39487)
0807 18:43:02.185478 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.185483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.186660 (+ 1177us) server_negotiation.cc:407] Connection header received
0807 18:43:02.188724 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.188729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.188785 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.188845 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.192578 (+ 3733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.193457 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.198572 (+ 5115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.198929 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.199023 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.204347 (+ 5324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.204352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.204355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.204388 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.206482 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.211048 (+ 4566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.211053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.211055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.211149 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.211865 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.211890 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.211892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.212129 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.212284 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.213337 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.213467 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:43:02.215828 9766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.184353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38626 (local address 127.6.138.126:42319)
0807 18:43:02.184486 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.184491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.186419 (+ 1928us) server_negotiation.cc:407] Connection header received
0807 18:43:02.187026 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.187030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.187087 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.187149 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.194614 (+ 7465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.195478 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.199205 (+ 3727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.199615 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.199707 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.207170 (+ 7463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.207175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.207178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.207211 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.209473 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.211473 (+ 2000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.211477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.211479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.211569 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.212619 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.212623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.212626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.212848 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.212990 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.215525 (+ 2535us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.215661 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 18:43:02.221374 9820 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e6ac9a4845974d62a9e63befe642b24a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.221544 9820 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [sys.catalog]: This master's current role is: LEADER
I20260807 18:43:02.221833 9851 mvcc.cc:205] Tried to move back new op lower bound from 7315981034257362944 to 7315981034182078464. Current Snapshot: MvccSnapshot[applied={T|T < 7315981034257362944}]
I20260807 18:43:02.222163 9768 sys_catalog.cc:455] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e6ac9a4845974d62a9e63befe642b24a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.222301 9768 sys_catalog.cc:458] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:02.222259 9820 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e6ac9a4845974d62a9e63befe642b24a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.222383 9820 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a [sys.catalog]: This master's current role is: LEADER
I20260807 18:43:02.223722 9793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e6ac9a4845974d62a9e63befe642b24a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ac9a4845974d62a9e63befe642b24a" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 42319 } } peers { permanent_uuid: "e102f711aa454cdda17da1efc9d8c69c" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 39487 } } peers { permanent_uuid: "08e7a5987293429ab83010e2ccda8669" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39473 } } }
I20260807 18:43:02.223855 9793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 18:43:02.223891 9558 tablet.cc:2369] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:43:02.224323 9862 mvcc.cc:205] Tried to move back new op lower bound from 7315981034257362944 to 7315981034182078464. Current Snapshot: MvccSnapshot[applied={T|T < 7315981034257362944}]
I20260807 18:43:02.224687 9863 mvcc.cc:205] Tried to move back new op lower bound from 7315981034257362944 to 7315981034182078464. Current Snapshot: MvccSnapshot[applied={T|T < 7315981034257362944}]
I20260807 18:43:02.226646 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:02.184662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50802 (local address 127.6.138.124:39473)
0807 18:43:02.184780 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.184784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.188536 (+ 3752us) server_negotiation.cc:407] Connection header received
0807 18:43:02.188666 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.188672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.188729 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.188820 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.205912 (+ 17092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.206690 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.207878 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:02.208187 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:02.208297 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:02.217424 (+ 9127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:02.217428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:02.217432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:02.217464 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:02.219472 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.225099 (+ 5627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.225103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.225107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.225219 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:02.225582 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:02.225586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:02.225588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:02.225828 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:02.225965 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:02.226355 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 18:43:02.226489 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:43:02.234176 9849 catalog_manager.cc:1347] Generated new cluster ID: ba7100ad3c4a4db388167ba97ec8c9c3
I20260807 18:43:02.234266 9849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:43:02.257834 9849 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:43:02.258992 9849 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:43:02.269769 9849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6ac9a4845974d62a9e63befe642b24a: Generated new TSK 0
I20260807 18:43:02.270773 9849 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:43:02.292891 9766 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.291299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38640 (local address 127.6.138.126:42319)
0807 18:43:02.291417 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.291422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.291752 (+ 330us) server_negotiation.cc:407] Connection header received
0807 18:43:02.291927 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.291931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.291985 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.292079 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.292752 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38640: BlockingRecv error: recv got EOF from 127.0.0.1:38640 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 18:43:02.292891 9764 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.291057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50816 (local address 127.6.138.124:39473)
0807 18:43:02.291198 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.291203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.291894 (+ 691us) server_negotiation.cc:407] Connection header received
0807 18:43:02.292096 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.292099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.292147 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.292206 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.292753 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50816: BlockingRecv error: recv got EOF from 127.0.0.1:50816 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 18:43:02.293177 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.291507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51822 (local address 127.6.138.125:39487)
0807 18:43:02.291647 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.291652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.292108 (+ 456us) server_negotiation.cc:407] Connection header received
0807 18:43:02.292330 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.292333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.292387 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.292448 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.293052 (+ 604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51822: BlockingRecv error: recv got EOF from 127.0.0.1:51822 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 18:43:02.296535 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.295547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51824 (local address 127.6.138.125:39487)
0807 18:43:02.295665 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.295669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.295708 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:43:02.295852 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.295855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.295933 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.296027 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.296423 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51824: BlockingRecv error: recv got EOF from 127.0.0.1:51824 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 18:43:02.300767 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.299780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.6.138.125:39487)
0807 18:43:02.299925 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.299929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.299943 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:43:02.300014 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.300017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.300065 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.300140 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.300671 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51838: BlockingRecv error: recv got EOF from 127.0.0.1:51838 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 18:43:02.307946 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.306921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.6.138.125:39487)
0807 18:43:02.307055 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.307059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.307227 (+ 168us) server_negotiation.cc:407] Connection header received
0807 18:43:02.307322 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.307325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.307378 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.307487 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.307840 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51852: BlockingRecv error: recv got EOF from 127.0.0.1:51852 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 18:43:02.315024 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.313856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.6.138.125:39487)
0807 18:43:02.314044 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.314049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.314174 (+ 125us) server_negotiation.cc:407] Connection header received
0807 18:43:02.314338 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.314341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.314397 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.314487 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.314916 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51854: BlockingRecv error: recv got EOF from 127.0.0.1:51854 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
W20260807 18:43:02.326167 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.325088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.6.138.125:39487)
0807 18:43:02.325210 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.325214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.325362 (+ 148us) server_negotiation.cc:407] Connection header received
0807 18:43:02.325483 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.325487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.325538 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.325632 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.326065 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51862: BlockingRecv error: recv got EOF from 127.0.0.1:51862 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 18:43:02.346033 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.344648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.6.138.125:39487)
0807 18:43:02.344805 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.344812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.345077 (+ 265us) server_negotiation.cc:407] Connection header received
0807 18:43:02.345206 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.345210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.345268 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.345361 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.345902 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51874: BlockingRecv error: recv got EOF from 127.0.0.1:51874 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 18:43:02.381860 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.380380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.6.138.125:39487)
0807 18:43:02.380543 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.380548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.380731 (+ 183us) server_negotiation.cc:407] Connection header received
0807 18:43:02.380906 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.380909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.380962 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.381029 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.381658 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51876: BlockingRecv error: recv got EOF from 127.0.0.1:51876 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 18:43:02.449576 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.448164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51888 (local address 127.6.138.125:39487)
0807 18:43:02.448366 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.448371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.448576 (+ 205us) server_negotiation.cc:407] Connection header received
0807 18:43:02.448766 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.448769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.448822 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.448919 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.449444 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51888: BlockingRecv error: recv got EOF from 127.0.0.1:51888 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:43:02.581287 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.580004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.6.138.125:39487)
0807 18:43:02.580210 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.580218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.580353 (+ 135us) server_negotiation.cc:407] Connection header received
0807 18:43:02.580500 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.580503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.580552 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.580644 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.581156 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51896: BlockingRecv error: recv got EOF from 127.0.0.1:51896 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 18:43:02.841147 9640 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:02.839764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51902 (local address 127.6.138.125:39487)
0807 18:43:02.839919 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:02.839923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:02.839994 (+ 71us) server_negotiation.cc:407] Connection header received
0807 18:43:02.840141 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:02.840143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:02.840196 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:02.840309 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:02.840982 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51902: BlockingRecv error: recv got EOF from 127.0.0.1:51902 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:43:03.028805 9790 catalog_manager.cc:1259] Loaded cluster ID: ba7100ad3c4a4db388167ba97ec8c9c3
I20260807 18:43:03.028944 9790 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c: loading cluster ID for follower catalog manager: success
I20260807 18:43:03.030668 9790 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c: acquiring CA information for follower catalog manager: success
I20260807 18:43:03.031847 9790 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e102f711aa454cdda17da1efc9d8c69c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:43:03.054081 9815 catalog_manager.cc:1259] Loaded cluster ID: ba7100ad3c4a4db388167ba97ec8c9c3
I20260807 18:43:03.054162 9815 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669: loading cluster ID for follower catalog manager: success
I20260807 18:43:03.056036 9815 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669: acquiring CA information for follower catalog manager: success
I20260807 18:43:03.057268 9815 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 08e7a5987293429ab83010e2ccda8669: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:43:03.359476 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:03.355326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.6.138.125:39487)
0807 18:43:03.355574 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:03.355580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:03.355681 (+ 101us) server_negotiation.cc:407] Connection header received
0807 18:43:03.355868 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:03.355871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:03.355999 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:03.356155 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:03.356470 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.357007 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.357878 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.358126 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.358235 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:03.358605 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:03.359072 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:03.359215 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:03.359252 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 18:43:03.359310 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260807 18:43:03.366225 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:03.362656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.6.138.124:39473)
0807 18:43:03.362877 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:03.362881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:03.362896 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:43:03.362983 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:03.362985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:03.363102 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:03.363205 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:03.363389 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.363914 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.364781 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.365004 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.365090 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:03.365439 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:03.365874 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:03.365970 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:03.366048 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 18:43:03.366100 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":82,"threads_started":1}
I20260807 18:43:03.373771 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:03.369387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.6.138.124:39473)
0807 18:43:03.369540 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:03.369545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:03.369952 (+ 407us) server_negotiation.cc:407] Connection header received
0807 18:43:03.370014 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:03.370016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:03.370065 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:03.370149 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:03.370607 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.371344 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.372487 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.372778 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.372884 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:03.373238 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:03.373313 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:03.373383 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:03.373541 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 18:43:03.373617 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:43:03.374002 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:03.369728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51916 (local address 127.6.138.125:39487)
0807 18:43:03.369864 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:03.369868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:03.370480 (+ 612us) server_negotiation.cc:407] Connection header received
0807 18:43:03.370671 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:03.370674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:03.370727 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:03.370774 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:03.371122 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.371722 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.372707 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.373020 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.373117 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:03.373441 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:03.373524 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:03.373617 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:03.373785 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 18:43:03.373852 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:43:03.374189 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:03.369324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38644 (local address 127.6.138.126:42319)
0807 18:43:03.369686 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:03.369691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:03.369712 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:43:03.369858 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:03.369864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:03.370040 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:03.370168 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:03.370430 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.371101 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.372348 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:03.372730 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:03.372858 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:03.373167 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:03.373513 (+ 346us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:03.373594 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:03.373970 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 18:43:03.374050 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":80,"threads_started":1}
I20260807 18:43:03.377856 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 9409
I20260807 18:43:03.391142 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 9533
I20260807 18:43:03.405409 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 9648
2026-08-07T18:43:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3545 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:43:03 dist-test-slave-2d19 krb5kdc[10002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:43:03 dist-test-slave-2d19 krb5kdc[10002](info): set up 2 sockets
Aug 07 18:43:03 dist-test-slave-2d19 krb5kdc[10002](info): commencing operation
krb5kdc: starting...
W20260807 18:43:04.655857 6697 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/test-user.keytab.
Aug 07 18:43:04 dist-test-slave-2d19 krb5kdc[10002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:43:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:43:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.138.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
Entry for principal kudu/127.6.138.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab.
I20260807 18:43:04.787930 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:41275
--webserver_interface=127.6.138.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:41275,127.6.138.125:33887,127.6.138.124:39593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:43:04.903787 10022 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:43:04.904209 10022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:43:04.904286 10022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:43:04.904351 10022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:43:04.909041 10022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:43:04.909127 10022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:43:04.909171 10022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:43:04.909207 10022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:43:04.909245 10022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:43:04.914594 10022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:41275,127.6.138.125:33887,127.6.138.124:39593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.126.keytab
--principal=kudu/127.6.138.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.126:41275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.138.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:43:04.915664 10022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:43:04.916743 10022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:43:04.925422 10030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:43:04.925455 10032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:43:04.925774 10035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:43:04.925635 10022 server_base.cc:1034] running on GCE node
I20260807 18:43:04.926362 10022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:43:04.927558 10022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:43:04.928753 10022 hybrid_clock.cc:648] HybridClock initialized: now 1786128184928725 us; error 45 us; skew 500 ppm
Aug 07 18:43:04 dist-test-slave-2d19 krb5kdc[10002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128184, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:43:04.942786 10022 init.cc:377] Logged in from keytab as kudu/127.6.138.126@KRBTEST.COM (short username kudu)
I20260807 18:43:04.943878 10022 webserver.cc:460] Webserver started at http://127.6.138.126:43797/ using document root <none> and password file <none>
I20260807 18:43:04.944536 10022 fs_manager.cc:359] Metadata directory not provided
I20260807 18:43:04.944638 10022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:43:04.944882 10022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:43:04.947038 10022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/data/instance:
uuid: "a32b69d9354e4ebea28a59cd74752e56"
format_stamp: "Formatted at 2026-08-07 18:43:04 on dist-test-slave-2d19"
server_key: "92fcab31c4647115e7afe03f555d41cb"
server_key_iv: "12936630c27f8c678814fcc8b38a2309"
server_key_version: "encryptionkey@0"
I20260807 18:43:04.947641 10022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/wal/instance:
uuid: "a32b69d9354e4ebea28a59cd74752e56"
format_stamp: "Formatted at 2026-08-07 18:43:04 on dist-test-slave-2d19"
server_key: "92fcab31c4647115e7afe03f555d41cb"
server_key_iv: "12936630c27f8c678814fcc8b38a2309"
server_key_version: "encryptionkey@0"
I20260807 18:43:04.951646 10022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:43:04.954308 10045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:43:04.955621 10022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:43:04.955763 10022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/wal
uuid: "a32b69d9354e4ebea28a59cd74752e56"
format_stamp: "Formatted at 2026-08-07 18:43:04 on dist-test-slave-2d19"
server_key: "92fcab31c4647115e7afe03f555d41cb"
server_key_iv: "12936630c27f8c678814fcc8b38a2309"
server_key_version: "encryptionkey@0"
I20260807 18:43:04.955876 10022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:43:04.990702 10022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:43:04.993948 10022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:43:04.994167 10022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:43:05.001360 10022 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.126:41275
I20260807 18:43:05.001691 10136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.126:41275 every 8 connection(s)
I20260807 18:43:05.002409 10022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/data/info.pb
I20260807 18:43:05.005250 10138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:43:05.006289 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 10022
I20260807 18:43:05.006433 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-0/wal/instance
I20260807 18:43:05.006801 6697 external_mini_cluster.cc:1428] Setting key b8d6811bee4e5b3fcd85ca157f776be1
I20260807 18:43:05.010408 10138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.138.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.125@KRBTEST.COM" created.
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128184, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:43:05.026680 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.012966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55132 (local address 127.6.138.126:41275)
0807 18:43:05.013425 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.013446 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.013730 (+ 284us) server_negotiation.cc:407] Connection header received
0807 18:43:05.014779 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.014783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.014976 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.015166 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.016002 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.017014 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.017993 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.018490 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.018622 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.021353 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.021358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.021369 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.021389 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.023513 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.024015 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.024024 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.024029 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.024201 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.024584 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.024591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.024595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.025174 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.025333 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.025713 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.025993 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":301,"thread_start_us":150,"threads_started":1}
I20260807 18:43:05.026680 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.012968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:41275 (local address 127.0.0.1:55132)
0807 18:43:05.013581 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.013619 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:05.014454 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:05.015162 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:05.015168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:05.015602 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:05.015846 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.015860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.017154 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.017158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.017876 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.017886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.018649 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.018652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.018748 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.019364 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:05.019381 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:05.021169 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:05.023634 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.023642 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.023651 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.023879 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.024335 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.024337 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.024339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.024450 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.025299 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:05.025305 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:05.025431 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 18:43:05.025584 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 18:43:05.025852 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":162,"threads_started":1}
I20260807 18:43:05.029284 10138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:05.030076 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.029726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:33887 (local address 127.0.0.1:37144)
0807 18:43:05.029847 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.030005 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:43:05.030301 10047 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
I20260807 18:43:05.031167 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.030976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:33887 (local address 127.0.0.1:37154)
0807 18:43:05.031073 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.031117 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 18:43:05.031319 10138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.125:33887: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab.
Entry for principal kudu/127.6.138.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab.
I20260807 18:43:05.039268 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.125:33887
--webserver_interface=127.6.138.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab
--principal=kudu/127.6.138.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:41275,127.6.138.125:33887,127.6.138.124:39593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:43:05.080616 10138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } attempt: 1
I20260807 18:43:05.081634 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.081288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:33887 (local address 127.0.0.1:37164)
0807 18:43:05.081478 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.081543 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:43:05.082419 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.082212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:33887 (local address 127.0.0.1:37172)
0807 18:43:05.082317 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.082363 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:43:05.082682 10138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.125:33887: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
I20260807 18:43:05.150864 10138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } attempt: 2
I20260807 18:43:05.151791 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.151464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:33887 (local address 127.0.0.1:37174)
0807 18:43:05.151639 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.151703 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:43:05.152652 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.152443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:33887 (local address 127.0.0.1:37180)
0807 18:43:05.152547 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.152593 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 18:43:05.152794 10138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.125:33887: Network error: Client connection negotiation failed: client connection to 127.6.138.125:33887: connect: Connection refused (error 111)
W20260807 18:43:05.155882 10151 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:43:05.156163 10151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:43:05.156235 10151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:43:05.156319 10151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:43:05.160502 10151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:43:05.160558 10151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:43:05.160578 10151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:43:05.160615 10151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:43:05.160647 10151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:43:05.166014 10151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:41275,127.6.138.125:33887,127.6.138.124:39593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.125.keytab
--principal=kudu/127.6.138.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.125:33887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.138.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:43:05.167096 10151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:43:05.168130 10151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:43:05.176826 10163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:43:05.176888 10151 server_base.cc:1034] running on GCE node
W20260807 18:43:05.176813 10159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:43:05.176813 10158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:43:05.177721 10151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:43:05.178937 10151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:43:05.180121 10151 hybrid_clock.cc:648] HybridClock initialized: now 1786128185180079 us; error 55 us; skew 500 ppm
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128185, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:43:05.182991 10151 init.cc:377] Logged in from keytab as kudu/127.6.138.125@KRBTEST.COM (short username kudu)
I20260807 18:43:05.183897 10151 webserver.cc:460] Webserver started at http://127.6.138.125:32865/ using document root <none> and password file <none>
I20260807 18:43:05.184583 10151 fs_manager.cc:359] Metadata directory not provided
I20260807 18:43:05.184679 10151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:43:05.184926 10151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:43:05.187024 10151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/data/instance:
uuid: "1b7e31d4f0534e319068e024981e566e"
format_stamp: "Formatted at 2026-08-07 18:43:05 on dist-test-slave-2d19"
server_key: "71baa3c264b8a9953a6ccf991e430a0e"
server_key_iv: "2bc8f84d8e0e8fb44f31384c166c5687"
server_key_version: "encryptionkey@0"
I20260807 18:43:05.187621 10151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/wal/instance:
uuid: "1b7e31d4f0534e319068e024981e566e"
format_stamp: "Formatted at 2026-08-07 18:43:05 on dist-test-slave-2d19"
server_key: "71baa3c264b8a9953a6ccf991e430a0e"
server_key_iv: "2bc8f84d8e0e8fb44f31384c166c5687"
server_key_version: "encryptionkey@0"
I20260807 18:43:05.191581 10151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:43:05.194219 10176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:43:05.195544 10151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:43:05.195712 10151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/wal
uuid: "1b7e31d4f0534e319068e024981e566e"
format_stamp: "Formatted at 2026-08-07 18:43:05 on dist-test-slave-2d19"
server_key: "71baa3c264b8a9953a6ccf991e430a0e"
server_key_iv: "2bc8f84d8e0e8fb44f31384c166c5687"
server_key_version: "encryptionkey@0"
I20260807 18:43:05.195833 10151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:43:05.211365 10151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:43:05.214313 10151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:43:05.214529 10151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:43:05.221810 10151 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.125:33887
I20260807 18:43:05.222116 10257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.125:33887 every 8 connection(s)
I20260807 18:43:05.222820 10151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/data/info.pb
I20260807 18:43:05.226042 10258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:43:05.226601 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 10151
I20260807 18:43:05.226737 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-1/wal/instance
I20260807 18:43:05.227072 6697 external_mini_cluster.cc:1428] Setting key 5b9089e84e9283bf1046e5b334692024
I20260807 18:43:05.231716 10258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.138.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.138.124@KRBTEST.COM" created.
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128185, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
I20260807 18:43:05.243976 10138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } attempt: 3
I20260807 18:43:05.250118 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.234698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:41275 (local address 127.0.0.1:55136)
0807 18:43:05.235517 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.235578 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:05.236719 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:05.237342 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:05.237349 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:05.237699 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:05.238198 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.238212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.239375 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.239377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.240402 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.240412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.240982 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.240993 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.241144 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.241687 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:05.241704 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:05.243752 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:05.246903 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.246913 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.246932 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.247320 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.247643 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.247650 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.247655 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.247853 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.248389 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:05.248398 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:05.248676 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 18:43:05.248869 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 18:43:05.249155 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":149,"threads_started":1}
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128184, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:43:05.251531 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.235754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55136 (local address 127.6.138.126:41275)
0807 18:43:05.235933 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.235941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.235962 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:43:05.236984 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.236992 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.237075 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.237191 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.238368 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.239238 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.240519 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.240787 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.240888 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.243942 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.243945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.243947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.243973 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.246782 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.247433 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.247436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.247438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.247534 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.247967 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.247971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.247972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.248211 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.250525 (+ 2314us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.251188 (+ 663us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.251336 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:43:05.252915 10258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:05.255577 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.244500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:33887 (local address 127.0.0.1:37186)
0807 18:43:05.244643 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.244659 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:05.244741 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:05.245631 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:05.245636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:05.245826 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:05.245968 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.245977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.246933 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.246940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.247530 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.247536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.248288 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.248295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.248449 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.248974 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:05.248990 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:05.250524 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:05.253927 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.253930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.253934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.254195 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.254503 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.254507 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.254509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.254600 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.255047 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:05.255050 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:05.255156 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 18:43:05.255263 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:43:05.255407 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 18:43:05.255688 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.244673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37186 (local address 127.6.138.125:33887)
0807 18:43:05.245022 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.245039 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.245070 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:43:05.245131 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.245140 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.245367 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.245515 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.246111 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.246804 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.247690 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.248125 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.248237 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.250671 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.250683 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.250698 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.250730 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.253763 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.254314 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.254320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.254322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.254408 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.254695 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.254700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.254702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.254936 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.255077 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.255375 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.255511 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":106,"threads_started":1}
I20260807 18:43:05.257582 10138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:05.258407 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.258144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33426)
0807 18:43:05.258273 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.258340 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128185, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:43:05.259207 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.258925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33434)
0807 18:43:05.259094 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.259149 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260807 18:43:05.259356 10138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39593: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.138.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab.
Entry for principal kudu/127.6.138.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab.
I20260807 18:43:05.262673 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.253447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:33887 (local address 127.0.0.1:37194)
0807 18:43:05.253651 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.253667 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:05.253745 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:05.254076 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:05.254079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:05.254272 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:05.254392 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.254402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.255180 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.255183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.255839 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.255851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.257020 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.257023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.257117 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.257585 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:05.257599 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:05.259194 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:05.261112 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.261117 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.261119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.261312 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.261648 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.261651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.261653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.261736 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.262174 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:05.262180 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:05.262296 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 18:43:05.262424 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 18:43:05.262562 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 18:43:05.262838 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.253478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37194 (local address 127.6.138.125:33887)
0807 18:43:05.253833 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.253842 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.253852 (+ 10us) server_negotiation.cc:407] Connection header received
0807 18:43:05.253902 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.253907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.253947 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.254042 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.254526 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.255044 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.255965 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.256855 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.257010 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.259371 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.259374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.259377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.259426 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.261003 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.261417 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.261421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.261422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.261550 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.261831 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.261834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.261836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.262047 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.262164 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.262520 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.262652 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":150,"threads_started":1}
I20260807 18:43:05.263218 6697 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
/tmp/dist-test-taskPYlYkN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.124:39593
--webserver_interface=127.6.138.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab
--principal=kudu/127.6.138.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.138.84:41955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.138.126:41275,127.6.138.125:33887,127.6.138.124:39593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:43:05.263531 10258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:05.264194 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.263953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33436)
0807 18:43:05.264059 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.264142 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 18:43:05.264434 10181 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
I20260807 18:43:05.265376 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.265121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33440)
0807 18:43:05.265247 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.265305 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:43:05.265553 10258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39593: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
I20260807 18:43:05.290511 10138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } attempt: 1
I20260807 18:43:05.291414 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.291115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33448)
0807 18:43:05.291270 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.291332 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:43:05.292146 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.291923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33456)
0807 18:43:05.292033 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.292082 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 18:43:05.292318 10138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39593: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
I20260807 18:43:05.314853 10258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } attempt: 1
I20260807 18:43:05.315824 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.315500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33464)
0807 18:43:05.315676 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.315740 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:43:05.316612 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.316397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33472)
0807 18:43:05.316502 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.316553 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 18:43:05.316743 10258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39593: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
I20260807 18:43:05.367478 10138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } attempt: 2
I20260807 18:43:05.368464 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.368102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33480)
0807 18:43:05.368321 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.368384 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:43:05.369252 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.369015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33486)
0807 18:43:05.369152 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.369198 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 18:43:05.369382 10138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39593: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
W20260807 18:43:05.383059 10276 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:43:05.383389 10276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:43:05.383455 10276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:43:05.383530 10276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 18:43:05.384899 10258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } attempt: 2
I20260807 18:43:05.385664 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.385380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33492)
0807 18:43:05.385522 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.385592 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 18:43:05.386363 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.386148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33500)
0807 18:43:05.386256 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.386301 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 18:43:05.386492 10258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.138.124:39593: Network error: Client connection negotiation failed: client connection to 127.6.138.124:39593: connect: Connection refused (error 111)
W20260807 18:43:05.387939 10276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:43:05.388034 10276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:43:05.388123 10276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:43:05.388200 10276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:43:05.388283 10276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:43:05.393822 10276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.138.84:41955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.138.126:41275,127.6.138.125:33887,127.6.138.124:39593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/krb5kdc/kudu_127.6.138.124.keytab
--principal=kudu/127.6.138.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.138.124:39593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.138.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:43:05.395004 10276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:43:05.396090 10276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:43:05.405035 10276 server_base.cc:1034] running on GCE node
W20260807 18:43:05.404985 10285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:43:05.404987 10283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:43:05.404980 10287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:43:05.405851 10276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:43:05.406970 10276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:43:05.408187 10276 hybrid_clock.cc:648] HybridClock initialized: now 1786128185408169 us; error 36 us; skew 500 ppm
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128185, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:43:05.411010 10276 init.cc:377] Logged in from keytab as kudu/127.6.138.124@KRBTEST.COM (short username kudu)
I20260807 18:43:05.412005 10276 webserver.cc:460] Webserver started at http://127.6.138.124:35929/ using document root <none> and password file <none>
I20260807 18:43:05.412657 10276 fs_manager.cc:359] Metadata directory not provided
I20260807 18:43:05.412755 10276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:43:05.412999 10276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:43:05.415108 10276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/data/instance:
uuid: "dadaddbd226b4ec5b838c429f36f435a"
format_stamp: "Formatted at 2026-08-07 18:43:05 on dist-test-slave-2d19"
server_key: "639d8424f82afa6049f73ba9876f4c21"
server_key_iv: "ed85139f9a1256c231615b03106ed15c"
server_key_version: "encryptionkey@0"
I20260807 18:43:05.415717 10276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/wal/instance:
uuid: "dadaddbd226b4ec5b838c429f36f435a"
format_stamp: "Formatted at 2026-08-07 18:43:05 on dist-test-slave-2d19"
server_key: "639d8424f82afa6049f73ba9876f4c21"
server_key_iv: "ed85139f9a1256c231615b03106ed15c"
server_key_version: "encryptionkey@0"
I20260807 18:43:05.419544 10276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:43:05.422263 10296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:43:05.423521 10276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:43:05.423676 10276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/data,/tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/wal
uuid: "dadaddbd226b4ec5b838c429f36f435a"
format_stamp: "Formatted at 2026-08-07 18:43:05 on dist-test-slave-2d19"
server_key: "639d8424f82afa6049f73ba9876f4c21"
server_key_iv: "ed85139f9a1256c231615b03106ed15c"
server_key_version: "encryptionkey@0"
I20260807 18:43:05.423794 10276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:43:05.439240 10276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:43:05.442363 10276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:43:05.442585 10276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:43:05.450102 10276 rpc_server.cc:307] RPC server started. Bound to: 127.6.138.124:39593
I20260807 18:43:05.450410 10371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.138.124:39593 every 8 connection(s)
I20260807 18:43:05.451196 10276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/data/info.pb
I20260807 18:43:05.454130 10374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:43:05.458154 10374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:05.461342 6697 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu as pid 10276
I20260807 18:43:05.461493 6697 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskPYlYkN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127881318083-6697-0/minicluster-data/master-2/wal/instance
I20260807 18:43:05.461802 6697 external_mini_cluster.cc:1428] Setting key 49b7ae0ed200d04a63dd1183ad45660b
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128185, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
I20260807 18:43:05.468544 10138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } attempt: 3
I20260807 18:43:05.472859 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.460184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55148 (local address 127.6.138.126:41275)
0807 18:43:05.460351 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.460355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.461268 (+ 913us) server_negotiation.cc:407] Connection header received
0807 18:43:05.462400 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.462404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.462477 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.462573 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.463767 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.464341 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.465348 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.465586 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.465704 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.468377 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.468380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.468382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.468406 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.470236 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.470904 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.470908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.470910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.470994 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.471407 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.471413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.471416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.471722 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.471817 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.472526 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.472660 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":56}
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128184, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.126@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
I20260807 18:43:05.474475 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.462494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33506 (local address 127.6.138.124:39593)
0807 18:43:05.462896 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.462910 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.462984 (+ 74us) server_negotiation.cc:407] Connection header received
0807 18:43:05.463155 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.463159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.463319 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.463486 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.464291 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.465119 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.466056 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.466392 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.466512 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.469749 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.469757 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.469770 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.469796 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.471685 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.472159 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.472165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.472167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.472308 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.472633 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.472637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.472639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.472971 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.473188 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.473605 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.473744 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"spinlock_wait_cycles":1792,"thread_start_us":214,"threads_started":1}
I20260807 18:43:05.474745 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.460661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.126:41275 (local address 127.0.0.1:55148)
0807 18:43:05.461122 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.461165 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:05.462132 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:05.462783 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:05.462792 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:05.463162 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:05.463619 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.463632 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.464454 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.464458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.465221 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.465231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.465740 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.465744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.465918 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.466552 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:05.466570 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:05.468215 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:05.470360 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.470371 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.470385 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.470795 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.471120 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.471123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.471125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.471289 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.471888 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:05.471893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:05.472160 (+ 267us) client_negotiation.cc:769] Sending connection context
0807 18:43:05.473106 (+ 946us) client_negotiation.cc:240] Negotiation successful
0807 18:43:05.473392 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":109,"threads_started":1}
I20260807 18:43:05.476421 10374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:05.477633 10258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } attempt: 3
I20260807 18:43:05.477741 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.469016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33510)
0807 18:43:05.469221 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.469234 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:05.469336 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:05.469655 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:05.469658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:05.469836 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:05.469913 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.469918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.470803 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.470806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.471597 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.471608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.472204 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.472208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.472316 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.472799 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:05.472814 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:05.474216 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:05.475995 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.475999 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.476001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.476281 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.476642 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.476646 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.476648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.476734 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.477194 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:05.477197 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:05.477296 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 18:43:05.477422 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:43:05.477554 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":21}
I20260807 18:43:05.478343 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.469118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.6.138.124:39593)
0807 18:43:05.469462 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.469471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.469483 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:43:05.469511 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.469513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.469544 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.469637 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.470029 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.470684 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.471713 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.472100 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.472190 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.474348 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.474351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.474354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.474376 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.475868 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.476407 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.476410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.476412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.476528 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.476831 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.476836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.476838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.477058 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.477199 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.477930 (+ 731us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.478167 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
I20260807 18:43:05.481734 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56: Bootstrap starting.
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128185, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.126@KRBTEST.COM
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128185, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.125@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
I20260807 18:43:05.487218 10138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56: Neither blocks nor log segments found. Creating new log.
I20260807 18:43:05.488334 10138 log.cc:824] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56: Log is configured to *not* fsync() on all Append() calls
I20260807 18:43:05.488674 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.478044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55154 (local address 127.6.138.126:41275)
0807 18:43:05.478156 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.478159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.478288 (+ 129us) server_negotiation.cc:407] Connection header received
0807 18:43:05.478400 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.478405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.478458 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.478523 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.479271 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.480007 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.481097 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.481396 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.481481 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.484534 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.484540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.484546 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.484577 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.486240 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.486887 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.486890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.486892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.487055 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.487499 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.487504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.487507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.487736 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.487875 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.488333 (+ 458us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.488516 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:43:05.489081 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.477077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.6.138.125:33887)
0807 18:43:05.477439 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.477445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.477459 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:43:05.477505 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.477510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.477558 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.477643 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.479001 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.479856 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.480892 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.481287 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.481375 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.483694 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.483698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.483700 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.483725 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.485674 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.486197 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.486201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.486204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.486292 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.487525 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.487529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.487530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.488139 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.488231 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.488592 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.488910 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":253,"spinlock_wait_cycles":1920}
I20260807 18:43:05.489291 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.478500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33518 (local address 127.6.138.124:39593)
0807 18:43:05.478668 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.478672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.478685 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:43:05.478720 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.478724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.478766 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.480522 (+ 1756us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.480610 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.481303 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.482349 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.482736 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.483184 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.485735 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.485741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.485746 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.485778 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.487370 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.487837 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.487842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.487846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.487924 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.488241 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.488246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.488249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.488482 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.488929 (+ 447us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.489020 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.489137 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"spinlock_wait_cycles":5888}
I20260807 18:43:05.489538 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.478025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33518)
0807 18:43:05.478268 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.478281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:05.478369 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:05.478915 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:05.478919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:05.479115 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:05.480414 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.480421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.481418 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.481422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.482196 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.482209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.482877 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.482882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.482979 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.484016 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:05.484032 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:05.485567 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:05.487496 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.487502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.487508 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.487737 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.488032 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.488035 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.488037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.488118 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.488628 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:05.488632 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:05.488730 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 18:43:05.488815 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:43:05.489336 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":1577}
I20260807 18:43:05.490937 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56: No bootstrap required, opened a new log
I20260807 18:43:05.492695 10258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e: Bootstrap starting.
I20260807 18:43:05.492921 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.476954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.125:33887 (local address 127.0.0.1:37196)
0807 18:43:05.477106 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.477119 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:05.477189 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:05.478586 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:05.478588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:05.478756 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:05.478858 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.478865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.479992 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.479994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.480770 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.480778 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.481524 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.481527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.481623 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.482050 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:05.482063 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:05.483550 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:05.485799 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.485802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.485804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.486089 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.486704 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.486709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.486712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.486812 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.488292 (+ 1480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:05.488295 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:05.488391 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 18:43:05.490201 (+ 1810us) client_negotiation.cc:240] Negotiation successful
0807 18:43:05.492692 (+ 2491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":1}
I20260807 18:43:05.494297 10374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:43:05.494901 10138 raft_consensus.cc:348] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } }
I20260807 18:43:05.495142 10138 raft_consensus.cc:729] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a32b69d9354e4ebea28a59cd74752e56, State: Initialized, Role: FOLLOWER
I20260807 18:43:05.495627 10258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e: Neither blocks nor log segments found. Creating new log.
I20260807 18:43:05.495477 10138 consensus_queue.cc:260] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } }
I20260807 18:43:05.496929 10258 log.cc:824] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:43:05.497166 10138 sys_catalog.cc:564] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.138.125@KRBTEST.COM
I20260807 18:43:05.499549 10258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e: No bootstrap required, opened a new log
Aug 07 18:43:05 dist-test-slave-2d19 krb5kdc[10002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786128185, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.138.124@KRBTEST.COM for kudu/127.6.138.124@KRBTEST.COM
W20260807 18:43:05.503364 10397 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:43:05.503448 10397 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:43:05.505390 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.494800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.138.124:39593 (local address 127.0.0.1:33522)
0807 18:43:05.495248 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 18:43:05.495263 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:43:05.495373 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:43:05.495897 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:43:05.495902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:43:05.496075 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:43:05.496159 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.496165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.497224 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.497227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.498051 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:43:05.498062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.498799 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.498803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:43:05.498881 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.499743 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:43:05.500213 (+ 470us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:43:05.501807 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:43:05.503898 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.503901 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.503903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.504129 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.504423 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:43:05.504426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:43:05.504428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:43:05.504496 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:43:05.504911 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:43:05.504915 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:43:05.504989 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 18:43:05.505080 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 18:43:05.505214 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285}
I20260807 18:43:05.505877 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.491128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37202 (local address 127.6.138.125:33887)
0807 18:43:05.492386 (+ 1258us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.492389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.492403 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:43:05.492438 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.492440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.492479 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.492561 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.493378 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.494153 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.495634 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.495974 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.496053 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.499364 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.499368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.499371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.499395 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.503479 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.504017 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.504021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.504024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.504134 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.504503 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.504506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.504508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.504727 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.504935 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.505614 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.505735 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1186}
I20260807 18:43:05.506134 10258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } }
I20260807 18:43:05.502426 10022 master_runner.cc:186] Error getting master registration for 127.6.138.125:33887: OK, instance_id { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" instance_seqno: 1786128185219717 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:43:05.506361 10258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7e31d4f0534e319068e024981e566e, State: Initialized, Role: FOLLOWER
I20260807 18:43:05.506450 10022 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:43:05.506215 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.494981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.6.138.124:39593)
0807 18:43:05.495071 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.495075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.495451 (+ 376us) server_negotiation.cc:407] Connection header received
0807 18:43:05.495481 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.495487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.495562 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.495643 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.496307 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.497111 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.498176 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.498486 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.498591 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.501937 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.501941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.501943 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.501965 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.503792 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.504225 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.504228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.504229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.504334 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.504592 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.504594 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.504596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.504790 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.504877 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.505761 (+ 884us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.505866 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 18:43:05.500221 10384 sys_catalog.cc:455] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.506605 10384 sys_catalog.cc:458] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:05.506727 10258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } }
I20260807 18:43:05.507826 10406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.508002 10406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:05.508867 10258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:43:05.509770 10374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a: Bootstrap starting.
I20260807 18:43:05.512709 10374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a: Neither blocks nor log segments found. Creating new log.
I20260807 18:43:05.513654 10374 log.cc:824] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a: Log is configured to *not* fsync() on all Append() calls
W20260807 18:43:05.514267 10427 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:43:05.514395 10427 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:43:05.514632 6697 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:43:05.515885 10082 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a32b69d9354e4ebea28a59cd74752e56"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55154
I20260807 18:43:05.516144 10082 raft_consensus.cc:482] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:43:05.516338 10082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:43:05.518368 10082 raft_consensus.cc:504] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } }
I20260807 18:43:05.516649 10151 master_runner.cc:186] Error getting master registration for 127.6.138.124:39593: OK, instance_id { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" instance_seqno: 1786128185448112 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:43:05.519044 10151 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:43:05.519425 10082 leader_election.cc:288] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [CANDIDATE]: Term 1 election: Requested vote from peers 1b7e31d4f0534e319068e024981e566e (127.6.138.125:33887), dadaddbd226b4ec5b838c429f36f435a (127.6.138.124:39593)
I20260807 18:43:05.520108 10214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a32b69d9354e4ebea28a59cd74752e56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1b7e31d4f0534e319068e024981e566e"
I20260807 18:43:05.520390 10214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:43:05.521384 10374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a: No bootstrap required, opened a new log
I20260807 18:43:05.521669 10214 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32b69d9354e4ebea28a59cd74752e56 in term 1.
I20260807 18:43:05.522085 10047 leader_election.cc:302] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7e31d4f0534e319068e024981e566e, a32b69d9354e4ebea28a59cd74752e56; no voters:
I20260807 18:43:05.522367 10384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:43:05.522627 10384 raft_consensus.cc:686] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [term 1 LEADER]: Becoming Leader. State: Replica: a32b69d9354e4ebea28a59cd74752e56, State: Running, Role: LEADER
I20260807 18:43:05.522994 10384 consensus_queue.cc:237] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } }
I20260807 18:43:05.524958 10384 sys_catalog.cc:455] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a32b69d9354e4ebea28a59cd74752e56. Latest consensus state: current_term: 1 leader_uuid: "a32b69d9354e4ebea28a59cd74752e56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.525082 10384 sys_catalog.cc:458] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [sys.catalog]: This master's current role is: LEADER
I20260807 18:43:05.525835 10374 raft_consensus.cc:348] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } }
I20260807 18:43:05.526087 10374 raft_consensus.cc:729] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dadaddbd226b4ec5b838c429f36f435a, State: Initialized, Role: FOLLOWER
I20260807 18:43:05.526381 10374 consensus_queue.cc:260] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } }
I20260807 18:43:05.530591 10442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:43:05.531560 10442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:43:05.537712 10214 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [term 1 FOLLOWER]: Refusing update from remote peer a32b69d9354e4ebea28a59cd74752e56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:43:05.538352 10384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:43:05.540901 10444 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.541090 10444 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:05.542322 10374 sys_catalog.cc:564] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:43:05.543292 10338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:43:05.543594 10339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a32b69d9354e4ebea28a59cd74752e56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dadaddbd226b4ec5b838c429f36f435a"
I20260807 18:43:05.544530 10406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a32b69d9354e4ebea28a59cd74752e56. Latest consensus state: current_term: 1 leader_uuid: "a32b69d9354e4ebea28a59cd74752e56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.544665 10406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:05.544881 10338 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [term 1 FOLLOWER]: Refusing update from remote peer a32b69d9354e4ebea28a59cd74752e56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:43:05.546320 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.522543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.6.138.126:41275)
0807 18:43:05.522670 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.522675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.523640 (+ 965us) server_negotiation.cc:407] Connection header received
0807 18:43:05.523830 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.523833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.523883 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.523952 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.525014 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.525796 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.527016 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.527360 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.527440 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.532375 (+ 4935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.532379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.532382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.532409 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.541266 (+ 8857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.541814 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.541817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.541819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.541935 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.542281 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.542284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.542286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.542534 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.542667 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.545996 (+ 3329us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.546141 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:43:05.546800 10339 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32b69d9354e4ebea28a59cd74752e56 in term 1.
I20260807 18:43:05.548153 10384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:43:05.551503 10438 sys_catalog.cc:455] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a32b69d9354e4ebea28a59cd74752e56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.551652 10438 sys_catalog.cc:458] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [sys.catalog]: This master's current role is: LEADER
I20260807 18:43:05.551857 10406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a32b69d9354e4ebea28a59cd74752e56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.552006 10406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:05.553433 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.522957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33526 (local address 127.6.138.124:39593)
0807 18:43:05.523076 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.523080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.523739 (+ 659us) server_negotiation.cc:407] Connection header received
0807 18:43:05.523982 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.523985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.524035 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.524085 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.525737 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.526444 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.528385 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.528740 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.528815 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.531791 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.531798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.531801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.531831 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.541439 (+ 9608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.547912 (+ 6473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.547916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.547918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.548019 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.552028 (+ 4009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.552032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.552034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.552303 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.552442 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.553152 (+ 710us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.553279 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:43:05.554014 10445 mvcc.cc:205] Tried to move back new op lower bound from 7315981047953461248 to 7315981047906000896. Current Snapshot: MvccSnapshot[applied={T|T < 7315981047953461248}]
I20260807 18:43:05.554960 10442 catalog_manager.cc:1347] Generated new cluster ID: 783439ba59504f4facfa52970d945d81
I20260807 18:43:05.555027 10442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:43:05.557718 10444 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a32b69d9354e4ebea28a59cd74752e56. Latest consensus state: current_term: 1 leader_uuid: "a32b69d9354e4ebea28a59cd74752e56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.557922 10444 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:43:05.560014 10438 sys_catalog.cc:455] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a32b69d9354e4ebea28a59cd74752e56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.560127 10438 sys_catalog.cc:458] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56 [sys.catalog]: This master's current role is: LEADER
I20260807 18:43:05.560532 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:05.536750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37212 (local address 127.6.138.125:33887)
0807 18:43:05.536881 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.536886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.537837 (+ 951us) server_negotiation.cc:407] Connection header received
0807 18:43:05.538203 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.538205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.538257 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.538329 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.545131 (+ 6802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.545896 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.546979 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:05.547300 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:05.547397 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:05.554995 (+ 7598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:43:05.555002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:43:05.555006 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:43:05.555037 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:43:05.556826 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.557294 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.557297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.557299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.557371 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:43:05.559596 (+ 2225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:43:05.559600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:43:05.559602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:43:05.559808 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:43:05.559899 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:05.560231 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 18:43:05.560373 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:43:05.561055 10444 sys_catalog.cc:455] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a32b69d9354e4ebea28a59cd74752e56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32b69d9354e4ebea28a59cd74752e56" member_type: VOTER last_known_addr { host: "127.6.138.126" port: 41275 } } peers { permanent_uuid: "1b7e31d4f0534e319068e024981e566e" member_type: VOTER last_known_addr { host: "127.6.138.125" port: 33887 } } peers { permanent_uuid: "dadaddbd226b4ec5b838c429f36f435a" member_type: VOTER last_known_addr { host: "127.6.138.124" port: 39593 } } }
I20260807 18:43:05.561149 10444 sys_catalog.cc:458] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a [sys.catalog]: This master's current role is: FOLLOWER
W20260807 18:43:05.563792 10463 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:43:05.563874 10463 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:43:05.563944 10448 mvcc.cc:205] Tried to move back new op lower bound from 7315981047953461248 to 7315981047906000896. Current Snapshot: MvccSnapshot[applied={T|T < 7315981047953461248}]
I20260807 18:43:05.566124 10466 mvcc.cc:205] Tried to move back new op lower bound from 7315981047953461248 to 7315981047906000896. Current Snapshot: MvccSnapshot[applied={T|T < 7315981047953461248}]
I20260807 18:43:05.607776 10442 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:43:05.609280 10442 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:43:05.632673 10442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a32b69d9354e4ebea28a59cd74752e56: Generated new TSK 0
I20260807 18:43:05.633647 10442 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:43:05.673337 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.669814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37224 (local address 127.6.138.125:33887)
0807 18:43:05.669941 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.669946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.672141 (+ 2195us) server_negotiation.cc:407] Connection header received
0807 18:43:05.672402 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.672405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.672473 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.672532 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.673200 (+ 668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37224: BlockingRecv error: recv got EOF from 127.0.0.1:37224 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 18:43:05.674893 10380 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.670310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33532 (local address 127.6.138.124:39593)
0807 18:43:05.670450 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.670456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.673850 (+ 3394us) server_negotiation.cc:407] Connection header received
0807 18:43:05.674057 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.674062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.674116 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.674169 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.674736 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33532: BlockingRecv error: recv got EOF from 127.0.0.1:33532 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 18:43:05.676378 10147 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.670789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.6.138.126:41275)
0807 18:43:05.670914 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.670918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.675395 (+ 4477us) server_negotiation.cc:407] Connection header received
0807 18:43:05.675622 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.675625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.675674 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.675729 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.676211 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55166: BlockingRecv error: recv got EOF from 127.0.0.1:55166 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 18:43:05.681299 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.679529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37238 (local address 127.6.138.125:33887)
0807 18:43:05.679639 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.679644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.680196 (+ 552us) server_negotiation.cc:407] Connection header received
0807 18:43:05.680470 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.680473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.680552 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.680609 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.681196 (+ 587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37238: BlockingRecv error: recv got EOF from 127.0.0.1:37238 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 18:43:05.686378 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.684869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.6.138.125:33887)
0807 18:43:05.684966 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.684969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.685460 (+ 491us) server_negotiation.cc:407] Connection header received
0807 18:43:05.685664 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.685666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.685718 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.685771 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.686279 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37246: BlockingRecv error: recv got EOF from 127.0.0.1:37246 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 18:43:05.692157 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.690581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37248 (local address 127.6.138.125:33887)
0807 18:43:05.690682 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.690688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.691191 (+ 503us) server_negotiation.cc:407] Connection header received
0807 18:43:05.691388 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.691391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.691458 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.691513 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.692053 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37248: BlockingRecv error: recv got EOF from 127.0.0.1:37248 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 18:43:05.701216 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.698843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37264 (local address 127.6.138.125:33887)
0807 18:43:05.698993 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.698997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.699740 (+ 743us) server_negotiation.cc:407] Connection header received
0807 18:43:05.699957 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.699977 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.700032 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.700091 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.701108 (+ 1017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37264: BlockingRecv error: recv got EOF from 127.0.0.1:37264 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 18:43:05.713392 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.711769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.6.138.125:33887)
0807 18:43:05.711896 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.711901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.712473 (+ 572us) server_negotiation.cc:407] Connection header received
0807 18:43:05.712665 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.712668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.712717 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.712767 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.713278 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37280: BlockingRecv error: recv got EOF from 127.0.0.1:37280 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 18:43:05.733978 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.732073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.6.138.125:33887)
0807 18:43:05.732197 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.732202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.732821 (+ 619us) server_negotiation.cc:407] Connection header received
0807 18:43:05.733043 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.733047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.733105 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.733159 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.733849 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37294: BlockingRecv error: recv got EOF from 127.0.0.1:37294 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 18:43:05.770118 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.768856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.6.138.125:33887)
0807 18:43:05.769040 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.769045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.769186 (+ 141us) server_negotiation.cc:407] Connection header received
0807 18:43:05.769295 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.769298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.769360 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.769470 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.769937 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37298: BlockingRecv error: recv got EOF from 127.0.0.1:37298 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 18:43:05.837657 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.836386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.6.138.125:33887)
0807 18:43:05.836549 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.836554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.836707 (+ 153us) server_negotiation.cc:407] Connection header received
0807 18:43:05.836892 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.836900 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.836949 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.837043 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.837544 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37300: BlockingRecv error: recv got EOF from 127.0.0.1:37300 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 18:43:05.969780 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:05.968247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.6.138.125:33887)
0807 18:43:05.968436 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:05.968442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:05.968605 (+ 163us) server_negotiation.cc:407] Connection header received
0807 18:43:05.968813 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:05.968817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:05.968887 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:05.968993 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:05.969626 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37306: BlockingRecv error: recv got EOF from 127.0.0.1:37306 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 18:43:06.229826 10272 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:43:06.228608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37312 (local address 127.6.138.125:33887)
0807 18:43:06.228740 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:06.228744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:06.228876 (+ 132us) server_negotiation.cc:407] Connection header received
0807 18:43:06.229038 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:06.229041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:06.229090 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:06.229153 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:43:06.229720 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37312: BlockingRecv error: recv got EOF from 127.0.0.1:37312 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:43:06.515756 10427 catalog_manager.cc:1259] Loaded cluster ID: 783439ba59504f4facfa52970d945d81
I20260807 18:43:06.515849 10427 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e: loading cluster ID for follower catalog manager: success
I20260807 18:43:06.517493 10427 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e: acquiring CA information for follower catalog manager: success
I20260807 18:43:06.518589 10427 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b7e31d4f0534e319068e024981e566e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:43:06.565310 10463 catalog_manager.cc:1259] Loaded cluster ID: 783439ba59504f4facfa52970d945d81
I20260807 18:43:06.565430 10463 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a: loading cluster ID for follower catalog manager: success
I20260807 18:43:06.567142 10463 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a: acquiring CA information for follower catalog manager: success
I20260807 18:43:06.568372 10463 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dadaddbd226b4ec5b838c429f36f435a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:43:06.748342 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:06.744241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.6.138.125:33887)
0807 18:43:06.744539 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:06.744546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:06.744597 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:43:06.744782 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:06.744786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:06.744923 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:06.745047 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:06.745325 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.745868 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.746752 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.746988 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.747095 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:06.747428 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:06.747880 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:06.748017 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:06.748081 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 18:43:06.748136 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":96,"threads_started":1}
I20260807 18:43:06.755086 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:06.751253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60826 (local address 127.6.138.124:39593)
0807 18:43:06.751503 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:06.751508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:06.751522 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:43:06.751678 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:06.751680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:06.751794 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:06.751945 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:06.752186 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.752727 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.753560 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.753798 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.753911 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:06.754225 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:06.754661 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:06.754801 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:06.754863 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:43:06.754921 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
I20260807 18:43:06.762354 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:06.758447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60836 (local address 127.6.138.124:39593)
0807 18:43:06.758559 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:06.758565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:06.758579 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:43:06.758624 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:06.758627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:06.758671 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:06.758767 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:06.759131 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.759802 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.761125 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.761397 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.761454 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:06.761877 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:06.761964 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:06.762062 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:06.762158 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 18:43:06.762207 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 18:43:06.762446 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:06.758017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.6.138.125:33887)
0807 18:43:06.758146 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:06.758150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:06.758581 (+ 431us) server_negotiation.cc:407] Connection header received
0807 18:43:06.758814 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:06.758817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:06.758876 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:06.758946 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:06.759358 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.760103 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.761117 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.761424 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.761533 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:06.761950 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:06.762035 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:06.762117 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:06.762266 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 18:43:06.762321 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:43:06.762926 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:43:06.757963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.6.138.126:41275)
0807 18:43:06.758224 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:43:06.758229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:43:06.758372 (+ 143us) server_negotiation.cc:407] Connection header received
0807 18:43:06.758640 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:43:06.758643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:43:06.758820 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:43:06.758945 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:43:06.759231 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.760084 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.761257 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:43:06.761619 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:43:06.761730 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:43:06.762058 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:43:06.762420 (+ 362us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:43:06.762511 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:43:06.762690 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 18:43:06.762753 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":89,"threads_started":1}
I20260807 18:43:06.766149 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 10022
I20260807 18:43:06.782271 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 10151
I20260807 18:43:06.796598 6697 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskPYlYkN/build/debug/bin/kudu with pid 10276
2026-08-07T18:43:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3393 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6939 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (305475 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST